smart auto-baud algorithm?

[ralphd]

Since the problem is >30 years old, I thought by now someone would have come up with the Holy Grail of autobaud algorithms.  Assuming 8N1 human readable ASCII, I think it is possible to detect the baud rate between 9600 and 115,200 after a single character.  None of the algorithms I have found take advantage of the fact that the 9th bit is always 0 and the 10th bit is always 1 for stop.  It also looks like the shortest run of ones or zeros is never more than 2 bit times, and there will always be an odd number of 0 or 1 bits, with the other count being even.

[John_ITIC]

I have worked on industrial serial communications auto-baud detection systems in the past. We sent a "spew" pattern of known data that was check-summed at the end. The receiver simply went through the available baud rates to see which baud rate resulted in valid data and check sum. If garbage was received, the receiver FIFO was flushed, the speed changed and the reception was attempted again. This was fairly simple so could also be implemented in an FPGA, up to several Mbit/second.

I think your actual implementation entirely depends on the protocol you are using on top of the basic serial protocol.

/John.

[ralphd]

I'm looking for a generic auto-baud algorithm.
Something similar to spritesmods, but that only eats the first character.
http://spritesmods.com/?art=autobaud&page=1

[coppice]

Autobauding when the mark space ratio of the signal is accurate is pretty easy. The problems come when the mark space ratio is distorted. FSK modems are particularly bad for this, but lots of channels used for serial data cause some distortion.

[hamster_nz]

Auto-bauding quickly off of readable characters is possible if you *assume* that the the shortest change you see is indicative of the bit time, and that characters are sent back-to-back.

Framing is harder. Imagine that you connected to a serial port using software flow control and saw the following bits streaming down the wire...

...01010101010101010101010101010100010101010101010101....

This is a valid sequence of bits for lots of 'U's in 8N1, with one 'Q', in the middle

...01010101 0101010101 0101010101 0100010101 0101010101 01....

it is also a valid sequence for 'U's with a 'T' in the middle.

...0101010101 0101010101 0101010101 0001010101 0101010101....

And it could also be taken as 'U's with an 'E' in the middle.

If you came in on the middle of that conversation you would be lost until something other than 'U's, 'T's, 'E's or 'Q's were transmitted (or the line became idle for a short while)....

[westfw]

I've seen "hardware autobaud" features described for some modern UART peripherals.
I'm not sure that it's an interesting problem on a microcontroller.  Just start the pins in GPIO mode and time the start bit.  If there's a specific autobaud character, you can confirm using the rest of the bits.  If not, you could consider timing the whole character...

the problem is >30 years old

It's not.  30 years ago, the problem was different (110 to 1200bps?)  Each time dialup modems got faster, the problem changed (you'll note that I'm being charitable making 1200bps available in 1985 :-))  We used to be happy with an autobaud that did 300-19200bps in two RETURNS (well, one, plus a second to make sure.)  Then 56k modems with compression came along, and 115200bps became a popular speed (and also, modems could configured the serial speed separate from the modulation/etc anyway, so you just locked them to the top comm speed, and made sure you had hardware flow control.)

The usual problem is not so much that you can't figure out the speed of traffic coming in based on the traffic, but that you don't have driver-level support, and you have to guess based on the traffic you get THROUGH YOUR EXISTING DRIVER AND OS.  After all, if you set the uart to max speed, you should get a unique sequence of characters and errors for any character at a lower speed.  But most drivers will discard the errors, and very few will report them in-sequence with the data.  (and, you may not have buffering turned on when those first characters arrive, causing you to miss some of them.

[mikeselectricstuff]

In practice I suspect that in many cases, problems due to mis-autobauding are probably more likely than the problem of just manually setting it, with trial and error if necessary.
At the very least you should probaby have a manually instigated "learn" mode, or be clever about when you attempt to autobaud as opposed to using the last value you saw.
Trying to do it through a UART is certainly fraught with problems due to loss of information and sometimes sketchy info on how a particular UART handles errors.
 
Probably the most reliable would be if you could sample bits at >2x the maximum rate, then apply the bitstream to a virtual UART at various baudrates and choose the one that gave the best match.

Of course if you can guarantee a fixed initial character it can be way easier. If that first character is a 00 byte you could autobaud over an 8:1 range with a single byte into a UART set to the lowest rate and count the number of ''0" bits in the result.

[bktemp]

If the auto-baud character is known, then it is easy. If not then it may not work at all:
The worst case are the characters 0x55 and 0xF0.
The first one transmits as 0101010101 the other one as 0000011111. If only one of these characters is transmitted continuously, it is impossible to tell if it the first one with the correct baudrate detected or the second one with 1/5 baudrate detected.
Most of the autobaud functions use 0x55 as the first character. It can also be used for FSK based transmissions followed by 0xFF to assure correct framing of the following data.

For a autobaud function in a logic analyser software I created a historgram of all bit lengths found and then tried to find the shortest bit length that matched the UART framing rules like no more than 10 zeros before at least 1 high is inserted. This is not really usable on a small microcontroller because it analyses all data before decoding it, but works really well in the PC based software.

[coppice]

I've seen "hardware autobaud" features described for some modern UART peripherals.

I think that addition has been driven the Lin bus. You really need hardware autobaud to support that efficiently.

[ralphd]

I'd like something like the spritesmods project that can plug into a standard USB-ttl dongle.  My current solution is to use a logic analyzer which does baud rate analysis, then set my terminal program to the right baud rate.
And that only works for standard baud rates unless I'm on linux.  A baud rate converter module would also allow reading the funky ~70kbps boot output of the esp8266 on computers that don't support that baud rate.

[mikeselectricstuff]

I'd like something like the spritesmods project that can plug into a standard USB-ttl dongle.  My current solution is to use a logic analyzer which does baud rate analysis, then set my terminal program to the right baud rate.
And that only works for standard baud rates unless I'm on linux.  A baud rate converter module would also allow reading the funky ~70kbps boot output of the esp8266 on computers that don't support that baud rate.

The FTDI drivers support funky baudrates via their COM port emulation interface - used to be that you had to mess with baud rate aliasing in the driver config but nowadays it seems to just work in most Windows apps at least. I regularly use 250K,1M 2M and 4Mbaud. there isn't much reason to use the oddball legacy rates these days - good internal RC oscs on MCUs mean that it makes more sense to use 'rounder' numbers for baudrate, e.g. 1Mbaud on a 16MHz osc

[ralphd]

Most of the autobaud functions use 0x55 as the first character. It can also be used for FSK based transmissions followed by 0xFF to assure correct framing of the following data.

When you have a device you are trying to debug or reverse engineer, and all you have is the UART pads on the PCB, an autobaud function that requres 0x55 for the first character is no help.

[ralphd]

there isn't much reason to use the oddball legacy rates these days [...]

Try telling that to the folks at Espressif.

[deephaven]

Most of the autobaud functions use 0x55 as the first character. It can also be used for FSK based transmissions followed by 0xFF to assure correct framing of the following data.

When you have a device you are trying to debug or reverse engineer, and all you have is the UART pads on the PCB, an autobaud function that requres 0x55 for the first character is no help.

If it was an activity like reverse engineering, I would hang a scope on the signal and measure the baud rate myself.

[FrankBuss]

When you have a device you are trying to debug or reverse engineer, and all you have is the UART pads on the PCB, an autobaud function that requres 0x55 for the first character is no help.

I've never seen any (modern) device which uses something different than 9600 baud or 115.2k baud, 8n1. And it is pretty easy to see in a terminal program that it is 115.2k if it is configured at 9600.

[mikeselectricstuff]

there isn't much reason to use the oddball legacy rates these days [...]

Try telling that to the folks at Espressif.

Sounds like they already know if it's using 70Kbaud

[ralphd]

When you have a device you are trying to debug or reverse engineer, and all you have is the UART pads on the PCB, an autobaud function that requres 0x55 for the first character is no help.

I've never seen any (modern) device which uses something different than 9600 baud or 115.2k baud, 8n1. And it is pretty easy to see in a terminal program that it is 115.2k if it is configured at 9600.

Check out the esp8266.

[bson]

The basic problem is that the levels are sampled on the line clock - it's not edge triggered.  This means RS-232C is immune to a lot of distortions - ringing, impedance mismatch reflections, peaking, bandwidth limiting, nonlinear phase effects.  It doesn't matter if each pulse "wiggles" or what it looks like; only its level at the edge of the line clock matters.  This line clock isn't embedded but it set out-of-band by some other means, which means there is an unknown - the one you're trying to solve for.

If you have a fast ADC you could sample the waveform and software decode it using a 2x progression of standard line clocks to see which one fits.  This requires an MCU where the RxD input pin can be temporarily configured as an ADC input.  (Actually, you could probably also do the same using it as a GPIO pin...)

[slateraptor]

Since the problem is >30 years old, I thought by now someone would have come up with the Holy Grail of autobaud algorithms.  Assuming 8N1 human readable ASCII, I think it is possible to detect the baud rate between 9600 and 115,200 after a single character.  None of the algorithms I have found take advantage of the fact that the 9th bit is always 0 and the 10th bit is always 1 for stop.  It also looks like the shortest run of ones or zeros is never more than 2 bit times, and there will always be an odd number of 0 or 1 bits, with the other count being even.

How do you figure? If your description is correct, then you've just described 7S1, not 8N1.

The basic problem is that the levels are sampled on the line clock - it's not edge triggered (1).  This means RS-232C is immune to a lot of distortions (2) - ringing, impedance mismatch reflections, peaking, bandwidth limiting, nonlinear phase effects.  It doesn't matter if each pulse "wiggles" or what it looks like; only its level at the edge of the line clock matters.  This line clock isn't embedded but it set out-of-band by some other means (3), which means there is an unknown - the one you're trying to solve for.

If you have a fast ADC you could sample the waveform and software decode it using a 2x progression of standard line clocks to see which one fits.  This requires an MCU where the RxD input pin can be temporarily configured as an ADC input.  (Actually, you could probably also do the same using it as a GPIO pin...)

(1) Sure it is, at the DCE; (2) false; and (3) if by "set out-of-band" you mean not physically wired in async setups, then sure.

Staying OT, hamster_nz and bktemp hit it spot on with respect to the OP...although I feel like the general discussion has somehow digressed by way of terminology confusion between baud and bit rate.

[Dago]

I can't remember where I saw this algorithm but something had an auto-baud algorithm and it was simply done by sending 'U' (known to both ends) and changing the baudrate until the receiver end got it right. Maybe some bootloader?

[westfw]

When you have a device you are trying to debug or reverse engineer, and all you have is the UART pads on the PCB

Ah.   Autobauding on substantial bursts of unknown text at non-standard rates is a significantly different problem than the one that people have been working on for 30 years...  :-(
Measuring the shortest pulse should get close pretty quickly; it's unlikely that the transmission would contain no odd (LSB==1) characters, and one of those would have a measurable start bit.

All bets are off if you get something like the old Tymnet sign-in prompt, which magically (?) displayed correctly regardless of whether you dialed in at 300bps or 110bps (or maybe even 1200bps?)

[Delta]

I've seen "hardware autobaud" features described for some modern UART peripherals.
I'm not sure that it's an interesting problem on a microcontroller.  Just start the pins in GPIO mode and time the start bit.

And how does one determine what's a start bit and what's any other Zero bit?

[hamster_nz]

I've seen "hardware autobaud" features described for some modern UART peripherals.
I'm not sure that it's an interesting problem on a microcontroller.  Just start the pins in GPIO mode and time the start bit.

And how does one determine what's a start bit and what's any other Zero bit?

One way to know for sure is if the line is idle for a short while (9 or 10 but-times) and then it has to be a start bit.

The only other way is if the data stream unambiguously proves it to be one. This can be quite hard to do in the presence of bit errors, as one in five bit errors will alter either a start or stop bit if the link is running at 100% throughput.