Release notes on the FSK modem/TNC for HF RTTY and SITOR

Distribution Contents

readme.txt     this file
relnotes.txt   release notes
info.txt       various information broadcasts of RTTY and SITOR stations
               giving times and frequencies; useful for testing
modem.asm      TASM source file, copiously commented and adorned
modem.obj      TASM object file, suitable for downline loading
modem.lst      TASM listing file

Include files from the DSP93 Distribution

regs.inc       TMC320 registers
ports.inc      DSP93 ports
serial.inc     16550 UART

16 June 1995

I think my RTTY/SITOR HF FSK modem may be ready for prime time. I've
done a lot of work on the basic algorithms and cleaned up phase-lock
loops, timing functions and logic. While a little fine tuning may be
indicated in the PLL time constants, the basic program appears to be
solid for both RTTY and SITOR operation, transmit-receive functions and
in all modes.

The carrier and noise-gate algorithms were refined to allow operation
right down in the mud, should the autostart false-alarm rate be
tolerable. The thing is incredibly sensitive - I can turn the audio gain
on my R72 all the way off and the modem will still copy the leakthrough.
The autostart algorithms were improved and properly integrated with the
synchronous modes. Transmitter flow control has been implemented using
hardware handshaking (UART CTS line), along with buffer overflow checks.
LED panel indications have been added to show the state of the CTS line
and overflow condition.

Little bugs were caught and fixed in such things as transmit RTS delays.
These are necessary in synchronous modes to avoid data loss at the
receiver, which takes a second or two to synchronize. Several bugs were
found and fixed in the SITOR decoder, which by the way now works like
gangbusters.

The readme.txt file has been expanded (!) with detailed explanations of
the algorithms, implementation and performance analysis. The performance
analysis is crude, incomplete and probably has errors. I'll fix it up
when I write a technical memo justifying why I spent much of the last
six months playing with radios and DSP programs, instead of research on
computer networks, which is what they pay me for. I hope and expect the
DSP-93 to become a useful lab tool.

The heatstroke condition noted previously has apparently been fixed with
the kind assistance of Jim Lynch, K4GVO. This project would not have
been possible without his help and that of the TAPR guys.

I would of course be delighted to hear from anyone finding this little
jewel of use, especially bug reports and performance comparisons. I
thought of putting a little noise generator in the code so the modem
could run performance curves on itself while I sleep. Not yet.

5 July 1995

My RTTY/SITOR HF FSK modem has been through another design review and
overhaul. The major objectives this haul are (1) a much improved RTTY
signal gate and autostart function, and (2) a much improved SITOR modem
with performance demonstrably better than existing analog designs, which
in my case is the AEA PK-232. While the PK-232 performance with weak
signal SITOR is commendable, my objective was to see how much better an
optimized linear receiver could do under suitably ugly band conditions.

I am happy to report the latest digital modem really and truly does
achieve good copy on signals the analog modem completely misses and, for
those signals that both modems can copy, the erasure rate with the
digital modem is considerably lower than with the analog one. I verified
these claims in tests with both modems connected to the same computer-
tuned SSB radio and SITOR signals transmitted by AT&T radiotelex
stations near New York, Miami and San Francisco, and signals transmitted
by the profligated WLO radiotelex station near Miami (heard on zillions
of frequencies in the NBDP coast station bands).

The computer tunes to one of these radiotelex stations twice an hour
throughout the day and to W1AW four times per day and records the
decoded text for both modems. On many occasions, SITOR signals were so
faint or so covered up in QRM that they could not be detected by ear
with the radio speaker, yet the digital modem copied them 100% and the
analog modem with varying degrees of garble. A typical example given in
the readme.txt file shows the digital modem produced an erasure rate
only one-fourth that of the analog modem when faced with a challenging
signal. These observations are of course very unscientific (we say
"anecdotal") evidence; true evidence requires controlled conditions,
measured error rates and comparisons with analytical models. Maybe I'll
do that sometime.

I learned a few things on the overhaul, mainly to avoid absolute
thresholds, where the amplitude of a signal (like the various distance
functions) is compared to a manually adjustable threshold. This means,
as the signal fades to the noise floor, the threshold cannot adapt, even
though the signal/noise ratio might be just fine. I changed the distance
function to be the ratio of the old distance to the RMS signal measured
over the bits of each character as it is decoded. The improvement,
especially with weak signals, is dramatic. In particular, the LED tuning
indicators work right down to the noise floor.

I also changed the character sync scheme to use all five repetitions of
the CCIR retrain signal - some 70 bits in all. The resulting processing
gain is so much that it is no longer necessary to calculate the
carrier/noise ratio or fiddle with signal gates, modulation gates and
related claptrap. The LED tuning display continues to work, but the
signal gate (LED 3) is not used. Turns out the PK-232 does not use the
threshold control in AMTOR either.

I then overhauled the RTTY decoder with what I learned in the SITOR
overhaul. The improvement in error rate is not as dramatic as with the
SITOR decoder, but certainly noticeable under weak signal conditions.
The major improvement comes from an overhaul of the carrier distance
code that operates the carrier gate. This resulted in a significant
increase in reliably distinguishing between a legitimate carrier and
noise. The implementation draws on technology developed in the area of
pattern analysis and classification, a weakness to which I succumbed
some years ago.

Responding to suggestions from current users, I added an unshift-on-
space function (u{0-3} command) and a suppress-erasure function (q
command). I happened to notice the radiotelex stations use the unshift-
on-space function on information broadcasts, but not on at least some
weather broadcasts. In fact, their weather broadcasts, apparently
relayed from U.S. Weather Bureau feeds, contain many phrases and
sentences separated by "...". An experienced RTTY operator would
certainly advise against that! Like the PK-232, the modem now prints an
erasure character "_" if a received character quality (distance) is
below the erasure threshold. I find this useful when adjusting the
thresholds, but it can be disabled with the q command toggle.
To conform with most radiotelex broadcasts, I switched the Baudot/ASCII
translation to use the CCIR keyboard, rather than the U.S. keyboard.
This conforms to W1AW broadcasts and AT&T broadcasts; but, on at least
some WLO broadcasts, a butchered U.S. keyboard is used, where the "#"
character apparently is replaced by "'". Ask not for whom the BEL tolls.

There are other additions to the command suite, including the v{+-} and
g{0-7} commands. The v{+-} command adjusts the erasure threshold, which
affects the "_" erasure as described above. The g{0-7} command adjusts
the analog gain of the DSP-93, which can be useful when connecting to
the recorder output on some radios. The a1 command is gone (replaced by
the a2 command), since the new carrier distance code is so good that it
is no longer necessary. As a side note, I don't presently plan to
implement more exotic operator features, such as command completion,
command confirmation or detailed status reports. Many of these functions
can be implemented in terminal program scripts or Unix shell scripts. I
do see a need to extract some performance data, such as error/erasure
rates, signal and noise density histograms, etc.

Meanwhile, for what it's worth, the distribution, including updated
readme.txt and relnotes.txt files, as well as new file info.txt
containing useful RTTY and SITOR broadcast frequencies and times, is on
louie.udel.edu in the pub/ntp directory as the files modem.zip and
modem.tar.Z. I'll upload the stuff to the TAPR site when I catch my
breath.

Last updated 5 July 1995

David L. Mills, W3HCF
Electrical Engineering Department
University of Delaware
Newark, DE 19716
302 831 8247 office, 302 831 9211 home
mills@udel.edu
