EEVblog Electronics Community Forum

Electronics => RF, Microwave, Ham Radio => Topic started by: profdc9 on October 13, 2021, 04:57:23 pm

Title: Proposal for another amateur radio data mode
Post by: profdc9 on October 13, 2021, 04:57:23 pm

Hello,

I have a proposal for a new HF amateur data mode.  It is intended for conversational use or contest QSOs.  It is intended to be simple enough to be implemented on microcontrollers such as an Arduino but also employ forward error correction in the form of the (24,12,8) extended Golay code.  It is called SCAMP (Simple Conversational Amateur Messaging Protocol).  It uses either on-off keying like CW or 2FSK like RTTY45, and so can use a simple crystal oscillator for OOK or a frequency synthesizer such as SI5351A for FSK, and a direct conversion receiver.  It is intended for HF digital communications on simple portable homebrewed transceivers, especially those that can operate from solar power or remotely.

I have attached my initial draft specification of the data mode.  Any comments would be appreciated, especially from those who have experience with coding theory.

73,
Dan
KW4TI

Title: Re: Proposal for another amateur radio data mode
Post by: RoGeorge on October 13, 2021, 06:55:00 pm

Not a ham, no idea about the proposal, just that there are a few name collisions already for the word "SCAMP".

Title: Re: Proposal for another amateur radio data mode
Post by: radiolistener on October 13, 2021, 08:56:44 pm

what is the reason for that?
There are already existing protocols like FT8 and other, and they already include error correction and designed to receive a weak singal in a noisy environment.

You can receive FT8 on microcontrollers such as STM32.

Title: Re: Proposal for another amateur radio data mode
Post by: MartinL on October 14, 2021, 08:33:43 am

The reasons are explained very clearly in the attached document...

FT8 is not practical on low end microcontrollers. Even the mcHF SDR, which is based on an STM32F4 and implements digital modulations like FreeDV onboard, doesn't have the power to implement FT8 decode. The STM32 implementation that's been demonstrated is on the STM32F7, a 200MHz Cortex-M7 part.

FT8 and similar also depend on having a fairly complex transceiver, capable of upper sideband modulation. What Dan is proposing here is viable for an 8-bit MCU connected to a trivial homebuilt transceiver like the Pixie.

I've read through the proposal. I don't really have the expertise to comment on the Golay coding scheme and error correction. I do think the mixed 6-bit/8-bit encoding approach could use some work, but that's a fairly minor detail at this stage.

Best thing would be to make it possible to try out! Perhaps implementing it in fldigi or similar.

Title: Re: Proposal for another amateur radio data mode
Post by: sv3ora on October 14, 2021, 09:59:09 am

When I see people asking the reason about something, I feel that they are so close-minded!

I have done something that is also intended for portable communications and simple gear. My mode does not use any MCU, not even any microchip!
It uses it's own "keyboard"as well as "display" so it is a complete system.

In a more elaborate system the "keyboard" can be replaced with many switches and trimmer resistors for each key.
The display can be replaced with a bargraph voltmeter (LM chip), with each LED to represent a letter.

It is shown here
http://qrp.gr/kiss703 (http://qrp.gr/kiss703)

If implemented using microcontrollers and filters, or computers, It can go down in bandwidth at the FT-8 levels, or even more.

Enjoy!
73
DE sv3ora

Title: Re: Proposal for another amateur radio data mode
Post by: UR5FFR on October 14, 2021, 12:09:38 pm

Hi, Dan

In "Digital encoding" chapter you write that "C is complement bit for the next bit". Why not generate C as inverse parity bit of next 4-bits group?

Title: Re: Proposal for another amateur radio data mode
Post by: profdc9 on October 14, 2021, 04:58:07 pm

I looked through the KISS703 description.  One thing that might be an issue is that 50% duty cycle or something very close to it must be achieved for the tone.  Otherwise there will be even harmonics and, for example, the 500 Hz tone generates a harmonic at 1000 Hz tone.  Depending on the tone length, the harmonics may be filtered out at the receiver because they are fall between other tone frequencies.  But its something to think about.  The easiest approach is to make sure the lowest and highest frequency used span less than an octave.

Perhaps you could save some bandwidth by using a scheme like DTMF to generate two tones at once?  The keyboard would be a 2-D matrix, and you could use a LM567 to detect only the tones that make up the tones generated by the matrix.

The "C" bit is intended to ensure that there are no more than 5 bits with adjacent mark or space intervals.  For example:

C XXXX C XXXX
0 1111 1 1110   would have 8 adjacent mark bits if the C bit was even parity
1 1111 1 1111   would have 10 adjacent space bits if the C bit was odd parity. 

If the C is the complement of the next bit

0 1111 0 1110  has four adjacent one bits
0 1111 1 0111  has five adjacent bits.

Making every fourth the complement forces a transition every fifth bit and therefore limits bit runs to five bits.

Title: Re: Proposal for another amateur radio data mode
Post by: UR5FFR on October 14, 2021, 09:03:34 pm

Hi

For 5 bits group exists only 16 combination which have 1/0 sequence length less than 3:

Code: [Select]

00100
00101
00110
01001
01010
01011
01100
01101
10010
10011
10100
10101
10110
11001
11010
11011

Any combination of these groups haven't sequence 0/1 with length more than 4 bits. By example in worst case

11011 11011 11011
00100 00100 00100

You can easily convert 4 bits to a group of 5 bits and vice versa using a lookup table.
This method detects 22.5% of single errors. Your algorithm is simpler to implement, but detects 20% of single errors. So 2.5 percent improvement  ;)

Title: Re: Proposal for another amateur radio data mode
Post by: sv3ora on October 14, 2021, 09:04:07 pm

Quote from: profdc9 on October 14, 2021, 04:58:07 pm

I looked through the KISS703 description.  One thing that might be an issue is that 50% duty cycle or something very close to it must be achieved for the tone.  Otherwise there will be even harmonics and, for example, the 500 Hz tone generates a harmonic at 1000 Hz tone.  Depending on the tone length, the harmonics may be filtered out at the receiver because they are fall between other tone frequencies.  But its something to think about.  The easiest approach is to make sure the lowest and highest frequency used span less than an octave.

Perhaps you could save some bandwidth by using a scheme like DTMF to generate two tones at once?  The keyboard would be a 2-D matrix, and you could use a LM567 to detect only the tones that make up the tones generated by the matrix.

The "C" bit is intended to ensure that there are no more than 5 bits with adjacent mark or space intervals.  For example:

C XXXX C XXXX
0 1111 1 1110   would have 8 adjacent mark bits if the C bit was even parity
1 1111 1 1111   would have 10 adjacent space bits if the C bit was odd parity. 

If the C is the complement of the next bit

0 1111 0 1110  has four adjacent one bits
0 1111 1 0111  has five adjacent bits.

Making every fourth the complement forces a transition every fifth bit and therefore limits bit runs to five bits.

This is an interesting idea and thank you for mention it. But DTMF is entirely different scheme.
You are right about the duty cycle. I have tried this thing and it works with the current oscillator in place. I mark the letters onto the meter without taking into account the duty cycle. If the circuit is made identical from all users, we do not have to worry about it, because they all have the same behaviour. The purpose of this experiment was to demonstrate what it is possible with just a few transistors, and hopefully it will interest these discrete minimalist builders. It was designed to do so.

In my next version I will use an LM331 as a voltage to frequency and frequency to voltage converter. With this chip, I might be able to reduce the bandwidth at much greater levels, so the harmonics of even the lowest frequency (the pilot tone), will be at much higher frequency than all the other characters of the KISS mode. I believe that I will be able to achieve much greater thermal stability with this chip. Yet in 8-pin through hole package.

I also have some great plans for a new display technology with memory, implemented in simple electromechanical ways, but I won't reveal this one yet.

I keep an eye on your mode for more improvements
73
DE sv3ora

Title: Re: Proposal for another amateur radio data mode
Post by: babysitter on October 14, 2021, 09:19:28 pm

Hi Dan,

I didn't see how you recover bitstream from received signal - what would be your favourite?
Some OPAMP Filter a la DJ6HP RTTY decoder, NE567, CORDIC on microcontroller?

Vy 73
Hendrik

Title: Re: Proposal for another amateur radio data mode
Post by: profdc9 on October 14, 2021, 10:08:26 pm

Quote from: babysitter on October 14, 2021, 09:19:28 pm

Hi Dan,

I didn't see how you recover bitstream from received signal - what would be your favourite?
Some OPAMP Filter a la DJ6HP RTTY decoder, NE567, CORDIC on microcontroller?

Vy 73
Hendrik

I plan to use a 3 pole LPF implemented on two op amps as an initial analog antialiasing filter.  Then I can use a relatively simple quadrature square wave demodulation without aliasing being too serious.  Basically, there are two square waves generated by the microcontroller that are 90 degrees apart.  Then the samples are added

+ + - - + + - - + + - - ...
+ - - + + - - + + - - + ...

etc. so that the demodulation only needs adds and subtracts.  The samples are retained over a finite window and are subtracted from the end of the window to make a FIR filter.   I am going to do this for five difference periodicities / frequency bands to be able to figure out where in the bandpass the signal is.  CORDIC should not be necessary if the frequencies being demodulated are close enough to Nyquist and the the 3rd harmonic of the square wave filter is sufficiently attenuated by the analog filter.

To put numbers to this, the filter will be from about 80 Hz-250 Hz, with about 40 dB/dec drop below 80 Hz and 60 dB/dec above 250 Hz.  The sample rate will be 2000 samples/s and the square waves will have periodicities of 8 (250 Hz), 12 (167 Hz), 16 (125 Hz), 20 (100 Hz), and 24 (83 Hz).   For example, the 8 periodicity signal has the following + and - FIR filter for a 72 sample window

+ + - -  + + - -  + + - -  + + - -  + + - -  + + - -  + + - -  + + - -  + + - - 
+ - - +  + - - +  + - - +  + - - +  + - - +  + - - +  + - - +  + - - +  + - - + 

so that its bandwidth should be 27 Hz.

The 8 periodicity signal will pick up some of the 24 periodicity signal, but the 12, 16, and 20 channels will mostly be used for data decoding, and the 8 and 24 will be used to find signals so that they can be centered in the 12, 16, and 20 channels by changing the LO.

I included the code I have written so far to implement this scheme if it would help to understand.  I also included the current schematic which shows the antialiasing filter.

Title: Re: Proposal for another amateur radio data mode
Post by: profdc9 on October 14, 2021, 11:05:05 pm

I looked at 4b/5b coding.

https://en.wikipedia.org/wiki/4B5B

The problem is that using this type of code with Golay codes makes it so that a single bit error in a 5b code results in multiple error bits in the decoded 4 bit code, resulting in a failed decode, or worse, an undetected error.  I am not sure how to get around this except test all code words with errors, and it is likely that some of them will accidentally decode because multiple bits will be flipped in the error table.  That is why I want a method that sends the Golay bits to be decoded unchanged.

Quote from: UR5FFR on October 14, 2021, 09:03:34 pm

Hi

For 5 bits group exists only 16 combination which have 1/0 sequence length less than 3:

Code: [Select]

00100
00101
00110
01001
01010
01011
01100
01101
10010
10011
10100
10101
10110
11001
11010
11011

Any combination of these groups haven't sequence 0/1 with length more than 4 bits. By example in worst case

11011 11011 11011
00100 00100 00100

You can easily convert 4 bits to a group of 5 bits and vice versa using a lookup table.
This method detects 22.5% of single errors. Your algorithm is simpler to implement, but detects 20% of single errors. So 2.5 percent improvement  ;)

Title: Re: Proposal for another amateur radio data mode
Post by: UR5FFR on October 15, 2021, 09:11:05 am

It seems to me that the issue of synchronization is not well developed enough. I am considering a situation where the listener can synchronize very quickly (like cw). Then the synchronization sequence will need to be inserted after each block, but it is long and this is ineffective. It would be nice to add a few bits to each block to make synchronization easier. In this case, a long synchronization sequence can occur at the beginning of the transmission. It can also be inserted into the stream periodically, but not often. Thus, if there are many errors in the channel, then the listener will wait for a long sequence to synchronize. And if there are few or no errors, then it will be able to synchronize quickly using additional information from the block.

Title: Re: Proposal for another amateur radio data mode
Post by: profdc9 on October 15, 2021, 01:59:29 pm

Quote from: UR5FFR on October 15, 2021, 09:11:05 am

It seems to me that the issue of synchronization is not well developed enough. I am considering a situation where the listener can synchronize very quickly (like cw). Then the synchronization sequence will need to be inserted after each block, but it is long and this is ineffective. It would be nice to add a few bits to each block to make synchronization easier. In this case, a long synchronization sequence can occur at the beginning of the transmission. It can also be inserted into the stream periodically, but not often. Thus, if there are many errors in the channel, then the listener will wait for a long sequence to synchronize. And if there are few or no errors, then it will be able to synchronize quickly using additional information from the block.

Thanks for taking the time to look over this.

So let me explain to you my thinking behind this.  I agree that there may be something lacking in the synchronization procedure.  CW inserts a longer pause than the pause between dit and dah to indicate the change to a new letter, and of course has one of the better signal processing devices available to decode it :) .   

I was thinking that like other modes such as RTTY45 or PSK31, the user would have to specify the base baud rate.  The receiver will already know the nominal period between possible bit transitions.  For example, if the base baud rate is 100 bits/s, then the receiver will nominally expect a transition to occur around 10 ms since the last transition was detected, or multiples of 10 ms.  Since only five zeros or ones can occur in a row, the maximum time between transitions is 5 bit intervals or 50 ms unless a resynchronization cycle is occurring.  The receiver can have a software PLL that runs at a nominal 100 Hz and has its phase updated (added or subtracted) when each new transition is detected to update when the next transition should be expected.   The phase would only be allowed to be updated a limited amount per bit, for example 0.1 ms, to account for drift between the clocks of the transmitter and receiver.  Then, for example, the fractional stability per bit required would be a maximum of 0.1 ms/10 ms or 1%, or for 5 bits would be 0.1 ms/50 ms or 0.2% or 2000 ppm.

Here I am relying on the fact that the microprocessor, especially if it uses a quartz crystal time standard, can generally have 100 ppm time accuracy or better.  For example, some (but not all) Arduinos use 16 MHz crystal (others use a ceramic resonator).  I am not sure if that is too much to require, since some microprocessors use RC internal standards or ceramic resonators.  But let's assume a crystal time standard of 100 ppm which should be readily achievable, and many crystals are significantly better than this.  Assuming the correct bit transition time can be established at the beginning of a message, two 100 ppm time standards can drift a maximum of 200 ppm relative to each other.  This corresponds to 0.002 ms in a 10 ms interval, and 0.01 ms between 50 ms transitions.  Therefore not too much correction per bit should be required.  Multipath, QRM/QRN, and fading will produce false transitions, so I am relying on the fact that the transmitter and receiver, if they have reasonably accurate time sources, once synchronized, should be able to stay tracked within a fraction of a bit interval over thousands of bits. 

Of course, initial synchronization is then very important.  What would be required to initially synchronize accurately is a bit pattern with an autocorrelation that has a sharp peak, so that the peak can be located accurately to establish the time standard.  For example, a Barker code ( https://en.wikipedia.org/wiki/Barker_code ) could be used.  However, this autocorrelation may be a computationally expensive operation.  A periodic bit pattern 111000111000 etc.  has a sharp autocorrelation peak but sidelobes corresponding to the mismatches of the pattern by one or more periods.  So as long as the receiver doesn't lose track of how many periods have elapsed in the synchronization signal, it should be ok.  It might be that three periods are not enough and more are required during synchronization to ensure a lock onto the transition edge.

I am open to adding bits between the 30 bit blocks but they would somehow need to be identifiable as separate from the blocks themselves.  I am not sure how to do this, but I will consider ways that it can be done.

Title: Re: Proposal for another amateur radio data mode
Post by: UR5FFR on October 18, 2021, 06:07:11 am

I have no idea how to improve synchronization.
It also seems to me that your algorithm will be sensitive to bursts of errors. Usually convolutional codes are used to correct them in combination with block code.