I have been looking for an affordable yagi antenna to use with a dual band handheld radio and I came across some very inexpensive VHF/UHF (HDTV) antennas. They are all 75 ohm though. Is it ok to use a 75 ohm antenna on a tx/rx radio that uses 50 a ohm antenna, or do I need to spend more money and buy a 50 ohm to 75 ohm converter?
You probably cannot use that antenna.
A 75 Ω-50 Ω mismatch is a SWR of 1.5, which is not intolerable for a handheld radio (which needs to tolerate a directly attached antenna being detuned by the body of the user or other objects). So the answer to the question in your title is "yes".
But, what you have posted is an amplified, receiving antenna for TV signals.
An "amplified antenna" has an amplifier which will amplify signals from the antenna to the feed line to the radio receiver. If you try to transmit through it, at best nothing will happen and at worst the amplifier will be destroyed.
(The amplifier might be a separate module that you can simply not attach; it depends on the design.)
Receiving is far more tolerant of impedance/SWR mismatch than transmitting so it's possible to declare an antenna is just "VHF/UHF"; transmitting needs a more precisely matched antenna. You will likely find that the antenna has a widely varying SWR which is very poor — for transmitting purposes — over most of the VHF range. It might happen that the antenna has a good SWR at the amateur 2-meter (144 MHz) and 70-cm (440 MHz) bands, but you should not expect it.
Even if the antenna did not have an amplifier, it might contain unsuitable contents. A dipole antenna connected to coaxial feed line needs a balun, and a balun intended only for receiving can be made with small, cheap components which cannot handle the power of even a handheld transmitter.