Is there any difference between demodulating LSB versus USB in software from IQ samples other setting the complex mixer frequency (LO) below or above the SSB signal frequency? (and excluding any filtering required)
No, there is no difference other than the relationship of the passband and the mixer/“carrier” frequency, as you have already observed.
- For USB, you downconvert (mix down) to baseband, apply a band-pass filter (with complex tap values) with edges at (more or less) 0 to +3 kHz, then ignore the I or Q output and take the other as the audio signal.
- For LSB, you do the same thing except that the band pass filter is around −3 to 0 kHz instead.
I'd also like to note that there really is not such a thing as “the SSB signal frequency”. Certainly you can point at the middle of the passband, more or less, but where that frequency lies depends on the filter shape used by the transmitter and would be very difficult to measure in the presence of noise. The only precise frequency value you can define for a SSB signal is the mixer/carrier frequency, and it's much more useful all around because that's the one that you have to get right to demodulate accurately.
Would swapping the I and Q channels before mixing and demodulation make any difference?
Swapping I and Q in any signal has the effect of inverting the frequency spectrum, which converts LSB into USB and vice versa, and a 90° phase shift (which is irrelevant to anything unless you are comparing it to the unswapped version in some way).
It'd be a perfectly fine way to implement a SSB receiver to define one filter and switch between LSB and USB by swapping I and Q or not. You can also negate Q instead.
It is useful to visualize what is going on: remember that a simple unmodulated signal is a complex sinusoid, or $(\sin(\omega t), \cos(\omega t))$. Visualize that: a point moving around the unit circle, or a helix in three dimensions. If you swap I and Q then you have $(\cos(\omega t), \sin(\omega t))$ instead, which is just a point moving in the other direction (the frequency is negated) and with a different position at any given time than the other case (the phase shift).