OK, let me try to answer this, but this answer may also be qualified as unqualified.
If you have a 50 Ohm receiver, and connect a perfectly (Z=R) 75 ohm antenna system, then your VSWR would be 1.5, and the "load mismatch attenuation" will be about 0.177dB. (with antenna system I include feedline)
I doubt that you would actually notice this.
However, you are using an RTL-SDR dongle with a source impedance of 75 ohm, and your feedline is 75 ohm, you use a proper 4:1 balun for a 300 ohm antenna, then you should be good to go. Even when you change over to a 50-Ohm receiver the loss due to mismatch is negligible.
The unknowns here is the suggested balun, and its behaviour when used with the OCFD. Unknown is also the quality and specifications of the "TV coax" you are planning of using.
As you are starting, and as you are only listening (no TX), I would actually fully agree with the answer given by Mike Waters unqualified or not.
Without knowing the very specifics of the TV-balun, and TV-coax, it would be hard for anyone to come up with a qualified or quantified answer for your particular question.
However, I did started off in this hobby as a SWL. And I have used unspecified components and material to build a range of antenna's.
- rule 1: any antenna is better than no antenna
- rule 2: any feedline is better than no feedline
- rule 3: it is a hobby in which you can go mad... watch your wallet
- rule 4: it is fun to experiment
For antennas I have used from rain gutters to water pipes, as feedline I have used from TV-coax to speaker-wire.
I have used a step-ladder as antenna...
Have I used TV-baluns ? -- yes I have, with various succes, pulled it down, swap in/out, try again
Have I used "some coax" ? -- yes I have, with various succes, sometimes it was just deaf... which means: replace.
Have I used a TV-balun with TV-coax and a OCFD ? -- no, I have no practical or qualified response to that...
As an SWL, work with the materials you have, work with what can be got cheaply, experiment, direct feed your OCFD, use the TV-balun... the succes rate is determined on what you hear, not what is calculated upfront. Then if you want to squeeze every single micro-dB out of your antenna system, then you can look at different materials/components, and budget for such.
And even when you calculate all this upfront, the moment you hang it outside in a tree or near a building, over real ground, the calculations may be off.
You do mention "may transmit", and just a word of caution here. If you really want to TX, then I would suggest to design/build an antenna for this with known components in order to avoid damage to your transmitter. This answer was written for RX not TX.
Have fun! It is a great hobby, and you will be surprised what you can receive with very little.