# Practical limits to maintaining coherence in an HF channel

Lately I've been thinking about weak signal communication on HF. Very weak, like weaker than even WSPR could achieve.

It would be nice if one could simply take any existing modulation and slow it down to achieve an arbitrarily high Eb/N0 and thus, given sufficient time, communicate with arbitrarily low power. However I understand ionospheric conditions introduce distortions which make this not really work in practice.

For example, there exists a WSPR-15 mode, which is like WSPR-2 but uses 15-minute instead of 2-minute intervals. This should mean WSPR-15 is about 9 dB more sensitive, but the documentation states:

WSPR-15 is not recommended for use at HF: the tone spacing is only 0.183 Hz, less than the Doppler spreading typical of many HF paths

So, what is "Doppler spreading", and how much of it is there on HF paths, and what can be done to overcome this challenge? More broadly, are there other properties of HF channels that limit the attainable sensitivity?

• Line-of-sight or ionospheric bounce? For bounced signals, one might want good statistics regarding multi-path phase noise (how much, changing how fast, how often, etc.) to characterize the channel for long duration semi-coherent signaling. – hotpaw2 Sep 16 '19 at 2:02
• I've never heard of Doppler Spread until now. – Mike Waters Sep 16 '19 at 20:32
• Phil, are you thinking of writing superior weak-signal software? If so, I believe that JT's source code is public, and maybe it'll give you some better ideas. Same thing for Linrad, that code is public. – Mike Waters Sep 17 '19 at 15:57
• Google "coherent cw" sigidwiki.com/wiki/Coherent_CW – Optionparty Sep 18 '19 at 1:42

Doppler spreading is the change in received frequency from a distance transmitter due to the rise and fall of the ionosphere along the signal path. When the effective height of the ionosphere rises, this lengthens the path and causes the received frequency to drop; when it falls the path decreases and the frequency rises.

You can measure this frequency change yourself in real time using simple equipment and, making some simple assumptions, compute the change in ionospheric height. The equipment and technique are both described in my Sept 2018 QEX article available here. The idea is to use a digital frequency synthesizer synchronized to GPS, then record the difference between the locally generated signal and a signal with a well-known frequency such as WWV. Then assuming the path is a simple triangular up-reflect-down profile, the change in frequency can be used to compute the change in path length and thence the change in ionospheric effective height.

My measurements suggest a 5 MHz frequency measured over a 1000 km path changes a few tenths of Hz during stable day and night periods, but can change up to half a Hz or more during twilight when ionospheric recombination (dusk) or excitation (dawn) is changing rapidly as the sun sets and rises over the path. These correspond to changes in the effective ionospheric height of a few tens of km.

If a transmitter is moving toward or away from you, the received frequency will get shifted up or down, depending on direction and rate of movement. Even if the transmitter and receiver are not moving relative to each other, but a reflector, reflecting the signal between them, is moving, the you can get the same Doppler effect.

It's well known (at minimum, from the license exam question pool) that non-line-of-sight HF propagation is made possible by refraction and reflection off of the ionosphere. But the ionosphere changes in many aspects, including height, not only with time-of-day, but with high altitude weather, solar radiation, and etc., lots of things. As the ionosphere altitude changes, you get a moving mirror, thus a bit of doppler shift of your HF signal frequency.

But that's not all. The ionospheric reflector is nowhere near flat. Thus you get multiple reflections (or refractive “bounces”), much like a fun house mirror. As the shape and layering changes, the directions and amplitudes of the different multi-path paths move around; and different combinations of paths constructively and destructively interfere in a (unpredictably?) changing pattern. Since each path has a different distance, its reflection likely has a different phase from other paths. Thus, depending on how the combinations of multiple reflection paths changes, you get phase modulation on top of frequency modulation of your signal. And fading with increasing phase cancellations.

If your demodulation scheme is using a DFT or FFT (or similar filter) on a strong signal, but half of the FFT window sees one phase and the other half sees the opposite phase, that signal will be invisible to the FFT result bin where you might expect to find your signal.

The statistics are such that the likelihood of a phase and frequency change of dF over time T increases with T. (I don't know where to find those statistics. Anybody?) There appear to be papers from the 70's and 80's on research in this area. Maybe earlier research papers as well.

So, any narrowband communication scheme should either:

1) track the doppler with a PLL or other adaptation, or

2) finish before the doppler shift and phase shifts are likely to be greater than the demodulation filter width and carrier lock delta-F.

wspr-2 likely finishes fast enough often enough. wspr-15 possibly might not over typical HF ionospheric paths. Neither wspr seems to have an internal PLL.

The equivalent of a PLL might be a signal re-acquisition. So perhaps repeating something the same length as a wspr-2 data transmission 7 or 8 times (or more) might provide more reliable coding gain than wspr-15, since each repeat would require a new fresh frequency and phase acquisition by the receiver, similar to a slow-motion step-function PLL.

Added: Here’s an ITU document recommending an HF channel simulation model that includes Doppler shift/spreading :

https://www.itu.int/rec/R-REC-F.1487/en

• What's the difference between the minimum signal WSPR can acquire, vs correctly demodulate? – Phil Frost - W8II Sep 17 '19 at 4:32
• Probability that some suspected waveform is more likely a wspr packet than just random noise. Vs. probability that a data decode might be correct. Lots of bits in a wspr packet are just to say “I’m not noise”, almost nothing to do with the message (callsign, etc) being coded/transmitted. – hotpaw2 Sep 17 '19 at 4:59
• If all you have are the first 2 minutes of a 15 minute transmission, it may be 9 dB below the MDS (because 2 is 9 dB less than 15). So how do you even know to suspect a waveform? – Phil Frost - W8II Sep 17 '19 at 15:46
• Good question. Perhaps one can't. Reducing the entropy of the waveform packet (coding less bits of information in it) can help make it more predictable, thus easier to make look different from random noise with some statistical significance. – hotpaw2 Sep 17 '19 at 16:51
• Yep, I think that's the idea generally. Putting less bits of information in the message is equivalent to sending the same message over a longer time. The problem is at some point I think making even an unmodulated carrier twice as long doesn't make it twice as easy to detect, and the question is what are the mechanisms behind this, and how can we quantify with some rigor, and is it possible to overcome such limitations and if so how? – Phil Frost - W8II Sep 19 '19 at 4:13

ITU Recommendation F.1487-0 defines methods for testing HF ionospheric paths for bandwidths up to 12 kHz. While ionospheric propagation can be complex, this document provides a starting point for the widely applicable basics.

It characterizes an HF channel with two parameters:

• multipath differential time delay, and

The multipath differential time delay is the maximum difference in time of arrival between multipath components. Put another way, it's the length of the channel impulse response. When the length of a symbol is very long compared to this value, differential time delay has negligible effect on demodulation performance. The ITU document states that the differential time delay exceeds 5 ms 5% of the time. Given that most very weak signal communication modes will have symbols much longer than this, differential time delay is not likely a major detriment to performance in this case.

The other parameter, Doppler spread, quantifies how "spread out" the power spectra of the signal will become due to each path having a randomly changing Doppler shift. The worst environment described is "disturbed conditions at high latitudes", with a Doppler shift of 30 Hz.

If the objective is coherently detecting a very long symbol, Doppler spread may better be understood by its dual, coherence time. Coherence time $$T_C$$ can be defined as:

$$T_c = {9 \over 16 \pi f_m}$$

where $$f_m$$ is the Doppler spread. This definition of coherence time gives the time where the correlation of the channel impulse response will be above 0.5. In other words, if one were to receive a signal at some time, and then an identical signal $$T_c$$ later, the correlation of those received signals will be on average 0.5.

For the worst case of 30 Hz, this works out to a coherence time of:

$${9 \over 16 \pi\ 30\:\mathrm{Hz}} = 5.97\:\mathrm{ms}$$

In other words, detecting a 6 ms symbol might work OK, but doubling the symbol length to 12 ms doesn't make the symbol twice as easy to detect since the second half of the symbol doesn't correlate perfectly with the first.

This is why polar paths are so challenging: Doppler spread can be extremely high.

WSPR-15 has a symbol rate of 0.1831 baud, whereas the ITU document gives a differential time delay of 0.5 Hz for "quiet conditions" at mid and low latitudes. From this we can already see the challenge: considered in the time domain, we can't count on an individual tone to maintain the same phase long enough that it won't start to cancel itself out. Or considered in the frequency domain, it's a challenge for WSPR-15 to resolve individual tones since the Doppler spread smears them together.

What can be done about it? I'm not entirely sure: I am after all answering my own question. But if the challenge is to establish communication even when slowing the symbol rate enough to approach the coherence time is insufficient, and transmitter power can't be increased, I'd guess the approach must be to take many shorter samples and add them noncoherently over a long time.

Consider the bad polar case where the coherence time is 6 ms: one could calculate an FFT every 6 ms and accumulate the magnitudes of each bin over a longer time. The Doppler spread means the received phase will be effectively random but a constant carrier, given enough time, will accumulate enough bias in the magnitude to become detectable above the noise. The short FFT duration will also mean the bins will be wider than necessary, which will introduce additional noise and require a wider tone spacing, but then if it was easy everyone would do it.

• (The TAPR Digital Communication Conference was this weekend; many talks covered fascinating topics related to this) Could it be as "simple" as sending on 2 nearby channels? (Okay, 3db power loss is bad, but maybe acceptable?) One channel with the 15 minute signal and 1 channel, say, 5 hz away with with a set of 15-ish seconds of pure tone ("someone is talking 5 hz down"). Could that +5 hz tone be used to estimate the current doppler shift? Could an appropriate amount of signal processing tease out the 15 minute signal? – Chris K8NVH Sep 23 '19 at 2:16

Any transmissive data channel will hit a lower power limit for readability when the signal no longer exceeds the noise floor of the transmission mode. This is another way of saying that the minimum-required signal strength for establishing data exchange is dependent on the position of the noise floor. So understanding the physics and math behind S/N ratios is needed to work on this problem.

In addition, it is at least theoretically possible to obtain very low noise interference if the signal bandwidth is extremely narrow. However, the narrower the bandwidth, the lower the theoretical maximum data rate will become, which means as the bandwidth approaches zero, so does the data rate. Working in this arena hence requires you to understand the relationship between bandwidth and data rate.

Data rate limits in narrow-band data transmission can be mitigated by data compression, in which some portion of the signal is discarded by an encoding algorithm so as to reduce the bandwidth needed to meet the data rate requirement, but there are information-theory based limits to how much compression can be applied to a signal before it stops being a signal. An information theory background will be helpful in this context.

Finally, the integrity of a transmitted signal can be enhanced in weak-signal applications by adding redundancy into the transmission- in the simplest form, this would mean transmitting every bit in the data stream twice to ensure it gets received at least once. In this simple model, adding full redundancy cuts the data rate in half. It is possible to do better than this using data encoding that incorporates things like checksum exchange to improve integrity without drastically decreasing the data rate. This falls into the realm of digital signal processing as practiced in the computer world, which is another thing you'd need to master in order to work in this field.

I invite the experts here to add their perspectives.

• The problem with the HF channel is not just AWN and signal loss, so the same methods of improving coding gain are likely not optimal, or perhaps even suitable. – hotpaw2 Sep 16 '19 at 15:13
• Thanks for the answer, but I think this misses the point. You write, "it is at least theoretically possible to obtain very low noise interference if the signal bandwidth is extremely narrow." The Shannon–Hartley theorem phrases this more rigorously. But in practice, on an HF channel, it's my understanding this doesn't really work because it's not an AWGN channel. That's what I'm asking about: specifically how is HF in practice not an AWGN channel, what bounds does that place on practical performance, and what methods are effective in addressing those challenges? – Phil Frost - W8II Sep 16 '19 at 20:13
• For example, WSPR-15 is theoretically 9 dB more sensitive than WSPR-2, but the manual says: "WSPR-15 is not recommended for use at HF: the tone spacing is only 0.183 Hz, less than the Doppler spreading typical of many HF paths." – Phil Frost - W8II Sep 16 '19 at 20:16
• I am sorry for missing the point. It sounds like you know a lot more about this than I do. Let me know if you think i should delete my answer. – niels nielsen Sep 16 '19 at 23:43

This is an interesting topic, as I operate CW only (for 60 yrs) , and now only QRP, and commonly copy signals down to SNR=1.

I certainly employ several different CW Filters which reduce the sideband noise power and limiter methods to quench QSB. ... Recently I did a research project, published on ResearchGate.net and also my own website www.GeoCities.WS/glene77is/ This project utilizes phased-filtering to produce two -60 dB notches surrounding the central f(0) band pass audio signal. This greatly helps attenuate the sideband noise power. Using this filter, I have consistently worked stations at SNR=1 and lower ( depending on just how much band-noise, static, QSB are present. ... But the OQ was not about CW nor "Navigating-by-the-Stars". rather the OQ was about computerized methods for Weak Signal Communication. ... So, I return to the original topic,
which was "weak signal communication on HF" using computerized methods ** .
... **To Mike Waters
, I like your suggestion about the 'original' error detection scheme which was the "ACK/NOACK" data exchange control, very popular in 1976 for me.
This Communication is still within the bounds of Father Nature > SNR=1. ...

(1) I would like your comments about a technique which comes from a QST article dated in 1976 (as I recall). --- Coherent CW was the name of the method. --- It required a timing co-ordination scheme (very un-attainable in 1976) . The original Coherent CW methods may be feasible ( 2019 ) using Global Position Time signals, but, by that same standard, would provide very slow and un-interesting communication to the current crop of ham operators. ...

(2) It appears to me that PSK-31 has incorporated this "clock" information in its phase-shifting (+/- 15.25 Hz) .
In our club, our operators have demonstrated accurate copy of PSK-31 at -9 dB. PSK-31 methods are useful for ordinary Weak Signal HF communication and the speed is aprox 33 wpm, and allows good two-way conversation style communication.
...

(3) It appears to me that the extreme of these techniques is the technology developed by "JT" for ham use (JT-65). A step further is a similar Weak Signal method employed to communicate with the several Space Probes. Space Probe signals are in the hundreds of dB below Johnson Noise levels. With these methods, DSP and statistical analysis is a key technique. Space Probe methods are not practical for ordinary use.
... This is an interesting topic. ... Glen Ellis, K4KKQ

As Neils mentions, I don't see any better technique other than incorporating checksum exchange. This would be true regardless of the modulation technique.

For example, both stations could be half-duplex and constantly exchanging ACK/NOACK data.

A NOACK[nowledge] would cause the data to be repeated until an ACK is received. An ACK[nowledge] from the receiving station would tell the transmitting station to send the next data packet.

• Since the OQ was edited, this no longer answers it. Oh well. – Mike Waters Sep 17 '19 at 15:52