overly large bandwith for a FSK signal?

I am trying to recover my 2FSK signal using an SDR. I have centered the SDR modem on the median frequency (the virtual carrier) and captured my signal via some 90 kHz bw modem. THis is the recording of it below. You can hardly make out the sine waves from it. I have highlit one such sine wave - 56 samples @ 2.048 MHz sampling rate = about 36.6 kHz, which is close to what I am emitting. Should the necessary modem's bandwidth be in this case 2 * 36.6 + 19.2 (baudrate) = around 90 kHz ?

I am getting pretty much the same results with a 200 kHz bw modem. However, with a 250 kHz modem the sines are much better. But not quite perfect, like in the picture below. Am I seeing this artifact because of my actual signal's BW being wider than the calculated BW and even wider than 250 kHz or is it something else that I am missing ?

edit 1 Previously the RTL AGC was turned on and the Tuner AGC turned off (with RF gain set to 48 - almost to the upper limit). The following two new captures are from 250 kHz modem with Tuner AGC turned on (the only difference from second capture above). Funny what I am noticing. Here how the beginning of the message looks like

and here is the end of it:

edit 2 MUch better looking now with AGC off. I had to turn off AGC to both RTL and tuner in order to be able to manually set it to a custom value. That custom value was the maximum: 49.6 db; my signal is pretty faint, At 16.6 db I was barely able to see it is there.

• at least the lower picture looks a lot like you have numerical clippling in there, where a number exceeding the maximum integer value becomes minimally negative, and the other way around. Can you try with reduced receiver gain? – Marcus Müller Jan 17 '17 at 17:24
• and: what is your SDR device? Also, I'm correctly assuming I'm looking at an IQ data stream, right? – Marcus Müller Jan 17 '17 at 17:29
• @MarcusMüller IQ data, yes, dongle is Nesdr: RTL2832U chipset and R820T2 tuner. Edited the question with further captures. thanks – kellogs Jan 17 '17 at 18:10
• this really still looks a lot like clipping.In your second recordings, the RTL AGC was on or off? Can you turn it off and set it to a low gain manually? – Marcus Müller Jan 17 '17 at 18:15
• @MarcusMüller many thanks! still does not look like a proper sine wave but I guess that is not that bad. I'd appreciate a few words on the tuner / RTL AGC vs manual setup issue in relation to my encountered situation - in an answer. – kellogs Jan 17 '17 at 19:39

So, I asked you to reduce the gain and disable AGC (auto-gain control), and you asked my to explain why :)

That had the effect of the RTL chipset no longer "overloading" its ADC with values that lead to negative values in the DSP chain (which is integrated in the RTL chip, I presume).

Normally, an AGC jobs is to adjust the gain so that exactly that doesn't happen. I don't know why yours didn't do that – but it's very probable that the RTL dongle's AGC , being designed for TV, is optimized to ignore single carriers (e.g. interference and, in analog TV, pilot tones).

With that out of the way:

Don't you have a pretty signal! That's extrodinarily clean.

So, this is the old complex signal and positive vs negative frequencies business.

Because you've set your receiver to the center between the higher and lower (FSK) subcarriers, we see them at $+f_\Delta$ and $-f_\Delta$. Now, negative frequencies only exist in complex signals.

A complex tone of frequency $f$ is defined as

$$e^{j2\pi f t}$$

with $j=\sqrt{-1}$ being the imaginary unit, and $t$ being time.

Then, Euler tells us:

$$e^{jx} = \cos (x) + j\sin(x)\text{ ,}$$

or applied to our harmonic signal

$$e^{j2\pi f t} = \cos (2\pi ft) + j\sin(2\pi ft)\text{ .}$$

Now, the cosine is the Real part, ie. what is visualized as the "upper" channel in your program, and the sine is the Imaginary part, what's visualized as "lower" channel.

Now, what's the difference between two signals at $+f_\Delta$ and $-f_\Delta$? Let's just compare:

\begin{align} e^{-j2f_\Delta t} &= \cos(-2\pi f_\Delta ) &&+ j\sin(-2\pi f_\Delta)\\ &&&\text{We know that \cos is symmetric, \cos(-x)=\cos(x),}\\ &&&\text{and that \sin is anti-symmetric, \sin(-x)=-\sin(x).}\\ &= \cos(-2\pi f_\Delta ) &&+ j\sin(-2\pi f_\Delta)\\ &= \cos(2\pi f_\Delta ) &&- j\sin(2\pi f_\Delta)\\\\ e^{j2f_\Delta t} &= \cos(2\pi f_\Delta ) &&+ j\sin(2\pi f_\Delta)\\ \end{align}

So, the difference between a positive frequency and it's negative counterpart is only that the sign of the imaginary part is flipped. Flipping a sign is nothing but a simple 180° phase shift.

That's exactly what's happening here.

Furthermore, because you can't know the phase of the complex tone compared to your receiver's oscillator, the real part isn't actually a sine (a sine has value $0$ for $t=0$, right) and the imaginary part not really a cosine, but both are shifted by a random phase; that means that when you jump between $+f_\Delta$ and $-f_\Delta$, both tones make a phase jump (and not just the imaginary part). To know whether the frequency is positive, or negative, is however pretty simple:

A cosine is simply a sine, but with a "phase advantage" of quarter a wave, or $\frac{2\pi}4 =\frac\pi2$. So, all you need to figure out at any point is "is the real part $\frac\pi2$ in front of the imaginary part, or the other way around?", and BOOM, there you go, FSK demodulator :)