/rant.

[paulca]

Sending 16 floats and a frame sync byte via UART.
00,00,00...  F8

Thumbs up, all good, no warnings, no errors, no fuss, all good.  Even watched the UART TDR register clocking 00's and even caught an F8 going out.

Working?

Hell no.  That's not what the chip sends.  It sends something completely different.  The scope sees this:

00,00,00... 10 FF

 

Maybe the scope is wrong.

Well the receiver is just as bad.  It reads the buffer about 50% of the time, errors the other 49% of the time and somehow gets itself out of a statemachine with all callbacks implemented and give up recieving.

... and when it does come back as successful.  It's just 0's.

This should be easy.  Why is it never actually straight forward?

All of this for "reasons", *

As there is no indication that any of this happening, both sides are absolutely sure they are sending and receiving it's just that both sides are making it up as they go along and neither realises it.

F8 was seen leaving the UART TDR register.  F8 is never seen on the scope and never by the receiver.  Where did it go?

This is like writing "Hello World" for god sake.  What does a newbie do when the STM32 variant gives him back  HHllllooa &^%&d! ?

[abyrvalg]

Baud rate/frame format mismatch? Are you using some built in UART decoder on the scope? Could you post a raw waveform instead of decoded values?

[rhodges]

This smells like something happens to the TX clock just after the TX empty flag. Does your TX empty interrupt do anything "interesting"?

[eutectique]

What do you see when you send F8?
Sending 00? Sending 00 F8? Sending 55? And AA?

[SiliconWizard]

Sending 16 floats and a frame sync byte via UART.
00,00,00...  F8

Thumbs up, all good, no warnings, no errors, no fuss, all good.  Even watched the UART TDR register clocking 00's and even caught an F8 going out.

Working?

Hell no.  That's not what the chip sends.  It sends something completely different.  The scope sees this:

00,00,00... 10 FF

 

Maybe the scope is wrong.

If you have a scope, can't you take a look at the actual UART signal and figure out why it interprets data in a way that is not what you intended?

[langwadt]

gotta show some code, simply sending data with the uart isn't rocket surgery; wait for for the tx empty flag, stick a byte in the data register

[cv007]

Maybe messing with the tx pin after the last byte put in txmit register, but not accounting for the fact a byte is still going out? The last byte gets corrupted and the receivers report what they see, including an additional start bit.

[paulca]

To be honest I spat the dummy out and went to eat something and watch a movie.

I took another quick look before bed on the scope and I see patterns.  Bunch of 0s then the 10FF (or 08FF) in LSB form.  However I did see several F8's go past.

My first check is going to be... what is in that memory I sent?  It shows me as just a set of 0's for the floats.  However, I have not checked the actual memory.  There may be struct padding etc in there.  The 10FF could be some form of padding and my believed length for that struct could be wrong and thus my receive buffer (nor scope decode) never sees a full struct.

The other thing to check is... does the Statistics_t*2 take longer to send than to generate and it's "clobbering" the memory while it's being sent.

On code, this is a basic generated HAL project.

Code: [Select]

typedef struct Statistics_struct {
float rms[8];
float peak[8];
char FRAME_SYNC;
} Statistics_t;
Statistics_t stats[2];

Statistics_t *busy_stats = &stats[0];
Statistics_t *available_stats = &stats[1];


Code: [Select]

HAL_UART_Transmit_DMA(&huart2, (uint8_t*)&stats[0], sizeof(Statistics_t) * 2);


Code: [Select]

void HAL_UART_TxHalfCpltCallback(UART_HandleTypeDef *huart) {
if (huart == &huart2) {
busy_stats = &stats[1];
available_stats = &stats[0];
return;
}
}
void HAL_UART_TxCpltCallback(UART_HandleTypeDef *huart) {
if (huart == &huart2) {
busy_stats = &stats[0];
available_stats = &stats[1];
return;
}
Debug_UART_TxCpltCallback(huart);
}
void HAL_UART_ErrorCallback(UART_HandleTypeDef *huart) {
Debug_UART_ErrorCallback(huart);

}
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart) {
Debug_UART_RxCpltCallback(huart);
}


Code: [Select]

bool handled = Analysis_Handle();
if (handled) {
haveHandled++;
}
if (haveHandled>10) {
haveHandled=0;
     // snip LED indicator code
available_stats->rms[0] = *output1L.active_rms;
available_stats->rms[1] = *output1R.active_rms;
available_stats->rms[2] = *output2L.active_rms;
available_stats->rms[3] = *output2R.active_rms;
available_stats->peak[0] = *output1L.active_peak;
available_stats->peak[1] = *output1R.active_peak;
available_stats->peak[2] = *output2L.active_peak;
available_stats->peak[3] = *output2R.active_peak;
}


[abyrvalg]

How can you be sure about bit alignment when you are reading from the scope? Do you count start/stop/parity(if any) bits? I.e. the 00 (with 8n1 config) will look like 0000000001, the last F8 will be 0000111111111…. (endless 1 until next packet). What about bit time periods? Post a waveform, too many things to guess otherwise.

[paulca]

The scope has a UART decoder.  It has the correct params.  115200/8/0/1.  It detects over/under runs, overlength bits etc.

The waveform for the full struct is too long to fit on the scope screen or it's decode list.

[paulca]

Something is fishy.  Even if we assume the 0x80 0xFF's are actually coming as part of the data (I don't think they should).

16 floats = 64 bytes.
char = 1 byte  (0xF8)

However 0xF8 to 0xF8 there are 67 bytes.  I can account for one, which may be the HAL quirk on DMA transfer sizes being length-1.  But I can't account for 2.  Nor can I keep 4 bytes alignment for floats that doesn't include the "char" somewhere.

There has to be padding or memory corruption.

Later I will replace the stats with "Hello World" and see how that looks.

[eutectique]

Nor can I keep 4 bytes alignment for floats that doesn't include the "char" somewhere.

If you place your floats at the beginning of the structure, they are guaranteed to be aligned at the boundary of float.

There has to be padding or memory corruption.

If you examine your structure in the debugger, what do you see? Is there a padding?

[eutectique]

The waveform for the full struct is too long to fit on the scope screen or it's decode list.

Is it 5 ms/division for the bottom trace?

[paulca]

Interesting.  Confers with the 67 bytes.  Yet I only count 65.

Memory looks "ok".


Encouragingly, the memory looks like it has the first 4 channels of each populated with decibel readings for peak and rms. 
Note, I gave the device actual must input, rather than letting it idle at 0.  0s were easier to see problems... and I did.


Leaves a few open questions though.

Where does the 08 FF come from?

Is the miss match in struct size padding/alignment around the 8 bit char?

Are the two answer above the same?

[paulca]

Oh.  My DMA transfer size is set to "byte", not word.  Is that likely to cause issues if the struct is padded around the char?

I'll try "Word" soon and see.

[paulca]

The waveform for the full struct is too long to fit on the scope screen or it's decode list.

Is it 5 ms/division for the bottom trace?

5ms top, 20uS bottom.  Although I'm not 100% if that's the divisions or the memory window.

[AVI-crak]

Very big nonsense to send a binary stream through UART. Not so - infinitely big nonsense!!!
  UART is designed to transmit text, any control character will drown in the binary stream.
Any number can be made hex with minimal effort.

Code: [Select]

char* hex_char(char* tail_txt, uintptr_t* value)//44
{
    int32_t tmp, jump; jump = 0;
    *tail_txt = 0;
    uint32_t vall = (uint32_t)value;
    do{
        tmp = vall & 0x0F;
        if (tmp > 9 ) tmp += 0x37; else tmp += 0x30;
        *(--tail_txt) = tmp;
        jump += 4095;
        vall >>= 4;
    }while ((vall )||((jump & 4096) == 0));
    *(volatile char*)--tail_txt = 'x';
    *(volatile char*)--tail_txt = '0';
    return tail_txt;
};

[paulca]

I have had that rumbling around in my head.

Not least that I personally hate binary protocols as they are a nightmare to debug.  If this was just the "Debug" UART statements I have peppered around... they work!  I can read them!

I was just being lazy avoiding the parsing and conversions.

[eutectique]

For the record, sizeof(Statistics_t) is 68.

[Siwastaja]

Absolutely nothing wrong with binary over UART.

The worst debugging nightmares I have seen have been with ASCII parsers, specifically those where the authors are not fully aware of all possible linefeed, implicit linefeed conversions due to existing systems, and how to write code that is robust regardless of any linefeed type (or conversions) used. That is entirely possible to do, just that even big companies like HP fail at it regularly.

Much more pain comes with deciding how to handle, if at all, backscpace and DEL controls. If not, there goes most of the human interoperability. If yes, that explodes parser complexity yet again. Whereas, the binary protocol has trivially simple parsers.

Binary is much slower to read and produce "by hand", but in all other respects, it's so much easier to deal with that I go with binary protocols a lot. This ease includes ease of debugging, due to having much simpler parsers.

[Siwastaja]

You simply forgot __attribute__((packed)). That is the enabler of simple binary protocols which do not need parsers at all. With the packed attribute, you can even do nasty and forbidden things like cast a pointer on top of any part of buffer, as the compiler won't assume alignment (works on all modern and sensible compilers). Does wonders on your codebase simplicity. Have been doing exactly that for almost a decade now and zero issues (regardless of what nctnico will say).

Another option is to parse manually for every field from/to a regular struct. It is still easier and less work than with ASCII, but still some work. I personally like packed structs a lot, another benefit is you can enjoy the agony of language purists clenching their teeth while they try to prove how your code fails, but it's rock solid.

You also forgot _Static_assert(sizeof(your_type) == 42, ""), where 42 is the placeholder for the size value you calculated manually. Double-checking, and using the language features to help automate that always pays off. A simple assert takes a minute to write, and possibly saves you from days of tedious debugging.

[hans]

Very big nonsense to send a binary stream through UART. Not so - infinitely big nonsense!!!
  UART is designed to transmit text, any control character will drown in the binary stream.
Any number can be made hex with minimal effort.

Sorry to put it; but thats Bullshit.

The framing includes a start bit, payload, stop bit and parity. The UART hardware has no clue what is text or a control character. That is interpreted by your terminal. It makes sense you cannot interpret binary streams via a regular terminal. Some terminal tools like moserial have a 2nd tab that allows you to see the binary data. There are tons of bootloaders that work over UART, and they will send binary streams to devices without problems. This has been tried and tested on literally millions of devices. It's easy to count that amount; Arduino, ESP32, etc. all use UARTs


To be honest, I wouldn't be surprised if the scope traces are bad. UART is async so the transmitter/receiver statemachines are synchronized based on historic transitions of the line, e.g. a start/stop bit. These transitions will be used to run an internal statemachine, and proper sync is essential for a good decode, even for ASCII data. If you drag the scope into a random section of your (burst) transfer with the front part chopped off, it's very likely you won't get a good decode.

You can't trust that a scope will try to start a decode at *any* position on the screen and then do some kind of error rate test to automagically figure out whats the right point to start. For scopes that only decode data that is presently visible on the screen, those decoders can be pure rubbish for async comms. I've got serial decode on my Rigol scope, but the UART is the one I use the least. I found it only works for a handful of characters that you can fit on 1 screen capture. If I want to capture more, I get a LA or (for UART) use a real serial cable. The only thing I do with a scope is verify timing.

Now for a hw UART receiver to work properly, I also assume it can properly lock on to the data stream with a sufficiently long idle stream. Is there a reason to continuously send UART data while the baudrate is only 115200?

[Siwastaja]

Arduino, ESP32, etc. all use UARTs

Besides, the legacy of binary UART protocols go much further than that. MODBUS from 1970's is a relatively simple binary protocol over UART. There is an ASCII version because someone thought it would be simpler, but no one uses or wants to use that. MODBUS has a lot of weird complications but using binary is not among them - the ASCII version shares all the oddities, plus the increased agony of having to use ASCII.

[paulca]

Absolutely nothing wrong with binary over UART.

The worst debugging nightmares I have seen have been with ASCII parsers, specifically those where the authors are not fully aware of all possible linefeed, implicit linefeed conversions due to existing systems, and how to write code that is robust regardless of any linefeed type (or conversions) used. That is entirely possible to do, just that even big companies like HP fail at it regularly.

Much more pain comes with deciding how to handle, if at all, backscpace and DEL controls. If not, there goes most of the human interoperability. If yes, that explodes parser complexity yet again. Whereas, the binary protocol has trivially simple parsers.

Binary is much slower to read and produce "by hand", but in all other respects, it's so much easier to deal with that I go with binary protocols a lot. This ease includes ease of debugging, due to having much simpler parsers.

I agree and disagree.  With a textual protocol you can fabricate test and control messages to help diagnose and address issues without having to have an interface which itself may be bugged or unavailable.  Examples would include JSON over REST versus say custom binary blob of socket.

The issues you describe regarding the various issues with text format and what exact that is are pretty much solved and available publicly.  Use an appropriate demonstrable pattern and format and use a well tested implementation of that.

However in MCU land, as we have discussed, nobody shares anything and to be honest. Who wants a "full" JSON parse or YAML parser in their micro?  Even I took short cuts with a piss poor performance serial tokeniser approach.

For every shortcut there will be bugs.

When you bring networks and operating systems in the loop, you run into many issues with the binary, such as host vs. network order and so on.

[paulca]

Arduino, ESP32, etc. all use UARTs

Besides, the legacy of binary UART protocols go much further than that. MODBUS from 1970's is a relatively simple binary protocol over UART. There is an ASCII version because someone thought it would be simpler, but no one uses or wants to use that. MODBUS has a lot of weird complications but using binary is not among them - the ASCII version shares all the oddities, plus the increased agony of having to use ASCII.

I think this highlights a point we agree on.  It's the bloody users.  If you dare to make an ASCII protocol, you absolutely must be prepared to handle ALL the human cases that the users will throw at it.  If you say, "The spec says upper case there.", they will answer, "Yes, but... "  If you make it binary, they haven't got a chance, don't know anything and are therefore less likely to know only enough to harm themselves or your lovely code.

My JSON parser doesn't support numbers because I was lazy, I made the strings.  If smoeone "out there" using my software expects sensible responses when he gives it "actual" JSON...  the fun may be all his.