I have yet to play with passive radar myself, but my impression is that you will need to deliberately set up one of your antennas so that it receives only the "signal directly from a transmitter" and another of your antennas so that it receives "a signal reflected off of some object".
Once you've done that, it should be clear which channel you have plugged which antenna into, no?
My guess here is somewhat corroborated by this diagram:
This particular diagram was found at https://www.researchgate.net/figure/A-passive-radar-system-with-a-reference-channel-and-surveillance-channel_fig1_283186198 from a published paper "Joint Delay and Doppler Estimation for Passive Sensing with Direct-Path Interference" by Xin Zhang and co-authors (DOI:10.1109/TSP.2015.2488584) but I'm seeing similar other examples in an image search for "passive radar reference channel".
Also corroborated by https://dopplerfish.com/passive-radar-hardware/ which says:
The observation channel antenna is a 5-element Yagi-Uda antenna […] The gain and directivity of the reference channel antenna is less critical for the radar’s performance, so we used a simple half-wave dipole.
This is somewhat opposite from what I first imagined but makes some sense especially since in this context: "The key was to select a location that provided an uninterrupted line of sight in the direction of the targets but provided some shielding from the direct path signal emitted by the [close and high-powered] radio tower."
Also corroborated by https://www.rtl-sdr.com/krakensdr-passive-radar-demonstration/ which uses two separate Yagi antennas, one pointed at the reference and one pointed in the direction of surveillance. (See https://github.com/krakenrf/krakensdr_docs/wiki/08.-Passive-Radar for more notes/tips, including "the reference signal should not be received directly by the surveillance antenna as much as possible".)
Based on https://en.wikipedia.org/w/index.php?title=Passive_radar&oldid=1112632179#Digital_beamforming and many of the pictures there of commercial systems, it looks like it's also fairly common (at least when cost is less of a concern) to use arrays of many separately-sampled antennas, which can be directionally focused electronically and/or digitally (rather than physically like a Yagi or dish).