Terminating unused ports will never make things worse, and indeed is necessary to provide "ideal" behavior. Is ideal behavior really necessary? It depends on the application. Is your setup working now? If so, terminators aren't necessary :-)
A typical splitter has these properties:
- It has an input port and two output ports, which we'll call A and B.
- It has high isolation, meaning power going into one output port doesn't show up at the other output port.
- It is matched, meaning the impedance looking in to any port is the same (usually one of 75 or 50 ohms).
- It is reciprocal, meaning it works in either direction: it can also be used as a combiner.
This sounds like nothing could go wrong: since the outputs are isolated then leaving one of them unterminated couldn't cause problems with the other output.
However, when the port is left open, it reflects all the power, and then you have to consider how that affects the other components. Even though the ports may be labeled "output", they are still inputs because the splitter is reciprocal. Power reflected from an output port is an input that must be considered.
For example, say 1 W enters the input port. One output is connected to a TV, which we assume is properly terminated internally. The other port is left open. (In practice the power is orders of magnitude less: I want to keep the numbers simple.)

simulate this circuit – Schematic created using CircuitLab
What happens:
- 1 W enters the input.
- The splitter divides that power evenly between the outputs. So 0.5 W comes out of each output.
- On port A, that power is converted to heat in the TV and we don't need to worry about it anymore.
- On port B, there's nowhere for the power to go so it's reflected. Now it's an input on "output B". Although it's called an "output", because the splitter is a reciprocal device, it works an an input, too.
- Half of the reflected power is lost as heat in the splitter.
- The other half is sent out the "input" port.
- Depends on what's attached to the input port!
If the input port is also 75 ohms, the reflected power is absorbed there, and we don't ever see it again. This reflected power means the source does not "see" 75 ohms, however that's unlikely to be an issue in practice for a TV system.
But if the input port isn't 75 ohms, the reflected power gets re-reflected and is sent another time into the input port. Now you have the opportunity for ghosting.
However in most cases, a number of factors combine to make this not that big of a deal.
- The source may not be 75 ohms, but it's not usually that far off. So the source does end up absorbing a significant fraction of the reflected power.
- The real world is already full of reflections. If the source is an antenna, it already contains time-shifted "ghosts" from the signal finding its way to you by multiple paths of differing length. An HDTV receiver employs channel equalization to compensate for this to some extent.