You can look at them. The characteristic impedance of a coaxial transmission line or connector is a function of the geometry of the conductors and the dielectric between them. Connectors are standardized, so for a given impedance they have a standard geometry. The pins are usually different sizes, and the amount of plastic dielectric in the connector might be different.
The specifics will vary based on the connector of course. A quick image search should turn up plenty of examples, for whatever kind of connector you have. Common ham radio connectors are BNC and N which come in 50Ω and 75Ω flavors. You might also run into a TNC connector, which is really just a BNC with a screw instead of bayonet shield.
"UHF" or SO-239/PL-259 connectors are also common. They were made in a time where "ultra high frequency" was much lower. They are fine on HF but less than optimal on VHF. They don't come in 50Ω or 75Ω flavors: there's just the one kind. They are basically shielded banana plugs and have a non-constant characteristic impedance throughout their length. Network analyzers didn't exist when they were designed, and that's why they don't work well on the upper parts of the VHF spectrum and above.
You could also measure the impedance with a network analyzer or time-domain reflectometer. But I'm guessing if you don't have an SWR meter, you won't have those things either.