UART Hardware fifo

[Psi]

The reason is that I want to learn the STM32 and I also want to learn inter-MCU communications. 

Normally, if you have to do inter-MCU communications, and you have say in how the communications is done, you don't pick UART.   However if learning is the goal then getting competent at using the UART is a must.

For real world Inter-MCU coms you usually pick either SPI, I2C, or you dedicate one 8/16/32 bit parallel bus between the MCUs with a couple extra IO for flow control.
One-directional parallel bus coms is easy, each MCU has 1 control IO output to the other MCU. The slave simply flips this bit when it has detected and read the data on the parallel bus. The master flips it's bit when it has detected the slave has read the data and it has put new data on the bus.

You can implement a nice parallel bus entirely in main() this way, without any interrupts and timers needed.
But you do need to guarantee, through design, that your main loop will always repeat fast enough for the minimum data rate you need. The amount of CPU overhead doing this is very small and the speed can be very fast due to being a parallel bus.  Of course you can add timers and interrupts to guarantee a specific data rate, if you want it.

[uer166]

Huh? Why would UART be inferior to i2c or SPI. I'd argue that a modern UART peripheral is just as easy to use as SPI, and substantially easier to use than i2c with less state space. It's also more robust to timing and rogue clocks since it's asynchronous.

Some peripherals even include extra nice stuff like address detection, break characters for framing, etc.

[Siwastaja]

UART is a very very typical choice for inter-MCU communication, nothing wrong or unusual about that.

I try to avoid I2C whenever possible because peripherals are always weird and/or buggy, I rather do multi-slave communication using UART + code where master polls and one slave at a time enable their TX pin. (Idea borrowed from typical RS485 communications.)

All interfaces have their quirks. Best about SPI is the hardware concept of packets, but not all peripherals do good job of exposing this possibility, so it's often a lost cause. UART always requires some effort to delimit packets, but it doesn't need to be rocket science.

[paulca]

Well.  It turns out, currently I have the following:

2 x UARTS (messaging and debug)
1 x SPI (Screen DMA transfer channel)
4 or so GPIO control channels (screen command channel, SPI CS/SS, buffer activity LED etc.)

The screen I currently have has no touchscreen chip, but there is one in the post.  The touchscreen channel used i2C.

And... If that's not enough of a birds nest of dupoint wires on the desk, the screen also has an SDCard slot on the back which could be used to pre-load icons/bitmaps etc....  I'm not sure what the interface is, it looks awfully like a direct connection to the SDCard pins, which sounds like a timing nightmare.

I'll be impressed if I get them ALL running at once, my STM32F411RE setup already has a number of yellow warnings due to partial conflicts.  I do have a free i2c, a partially free SPI and for the SDCard I'm not sure if the MCU has the correct peripheral to drive an SD card port directly... and if it does if there is capacity to enable it.

[Siwastaja]

One of the benefits in SPI is you can get to higher bitrates, which is fully thanks to the fact SPI peripherals do not do multisampling & data filtration internally. The flipside of this is, especially when you use high drive strengths and expect high bitrates, Signal Integrity becomes more important than on UART. UART peripheral does filtration for you, correctly detecting start bits under noise, and averaging multiple readings of data together, possibly setting some "noisy data" warning flag for you. On SPI, if you route the signals without consideration to return currents or coupling, for example over a cable, or in noisy environment, the risk of getting a false clock pulse (or plain old corrupted data bit) every now and then is much more real.

This is of course a solvable problem, just series-terminate the SPI bus and route it properly over a ground plane, world is full of interfaces susceptible to signal integrity issues (classical parallel memory buses are the same as SPI in this regard), but this is something to be aware of.

UART can go up to a few megabits per second, and if this is enough for you, then keep using it.

[paulca]

I might go to software engineering hell for this.  JSON parsing guerilla style.

I'm find it difficult to find homes for char*'s I can't just leave them in the buffer** and pass pointers around 'willy nilly'.

Then again.  I could hold onto a single message buffer, long enough to extract the value, format it, call the lcd->draw_string and then I don't need the char* again.  It puts a lot into one buffer lock, but I'll see.

/**
 * Extracts the "value" from the buffer provided.
 */
void payload_to_kv( uint8_t* buffer, uint16_t length, const char* key, key_value_t* kv){
   for(uint8_t *chr=buffer; chr<buffer+length; ) {
      if(*(chr++)=='"'
         && *(chr++)=='v'
         && *(chr++)=='a'
         && *(chr++)=='l'
           && *(chr++)=='u'
         && *(chr++)=='e') {
         while(  chr<buffer+length &&
               (*chr == '"' ||
               *chr == ':' ||
               *chr == ' ')) {
            chr++;
         }
         uint8_t* value_start = chr;
         // we hope to be at the start of the value.
         // lets find the end.
         while( chr<buffer+length &&
               *chr!='"' &&
               *chr!=',' &&
               *chr!='{' &&
               *chr!='}'
               ) {
            chr++;
         }
         uint8_t value_length = (uint8_t)(chr-value_start);
         // where do I put it?
         memcpy(value_start, ?, value_length);
         kv->value = ??;
         kv->key = key;
         return;
      }
   }
}

[Siwastaja]

Have you thought about data integrity? JSON is a high-level data format expecting flawless transport layer under it, any corruption totally wrecks havoc.

UART seems like very simple interface for such ASCII-based protocol, just JSON over UART, easy to write specification for: "send JSON over UART", job done. But really, you should consider adding some kind of checksum, and then this will mean another layer below the JSON. It can be very simple, like CRC16 over the JSON data after the last closing }, but still something the end-user of the protocol needs to consider, making it a tad less approachable. Especially when nasty things like CR-LF conversions are happening, my biggest pet peeve in poorly though out (nearly all!!) ASCII-based protocols.

Regarding that, people like nctnico say, "use ASCII protocol because it's easy to use and debug", yet I think I have never seen an actual product (besides ones by myself) which Just Work out of box whether you feed it CR, LF, CRLF or LFCR. In fact, just finding a serial communication program for linux and/or Windows, where you can configure the line feed conversions (so that the feature is actually fully implemented and tested, and works), is a real struggle. Last time I spend some 2-3 hours finally finding picocom and compiling the latest version from sources, because the version the distro offered couldn't really do this simple thing. I'm baffled by this constantly reoccurring struggle, because it's quite easy to get right; either do not use meaningful whitespace at all (for example, use ; to finish commands), but if you do want to use the enter/return key as a delimiter, accept both CR and LF as if they were exactly the same symbol, and make sure the parser doesn't go haywire with an empty message (i.e., repeated CR/LF delimiter).

[paulca]

I agree in a lot of ways.  It's why I said I should go to software engineering hell for writing code like that!

I'm just enjoying "spreading my wings" again, it's been a while of admin->configuration->wait in work.  Wrote about -44 lines of code in the past 2 weeks.   I  deleted some junk.  And too many years of nothing but JavaEE, JavaMAXEE, JavaMAXEE_ULTRA.  I have told management in work I find being called a "Senior or Principle Java engineer" an "offensive misappropriation of my talent".  I'm still waiting to see how that turns out!  This is cathartic therapy for a 20 year veteran of the industry, turning 48 in about 3 hours. 

IF the government promise holds up and I don't win the lottery I have close to 20 years to go.  I might keep an eye on the professional IoT space.

[paulca]

<rant>
Enterprise Java programming IRL these days is more like Arduino than Arduino is like C of old days.  It's not software development.  It's solution integration.  It's taking two (or more "sketches") and combining them into an application.   Not always, but A LOT!  (ETL jobs)

Spring BOOT REST APi == Arduino Blinky.

It's the same fucking thing!

Spring =  Arduino for Java.  ( oh.... I feel the heat now! )

Spring-webflowbollox*  = a result of "I need my I2C device supported under Arduino framework"
Spring-* libs, all 300Mb of them, you'll need them too and their deps.

You can just s/Arduino/Spring/ and it works.

I just love how in C  you are much more fully aware of what you have and where you are.
</rant>

[mikeselectricstuff]

Something to watch out for on UARTs with FIFOs is how they deal with the situation where some bytes arrive, but not enough to fill the FIFO to the point that it generates an interrupt. e.g.you may receive a string of data, but not see the last few bytes until the following massage starts.
I'm not familiar with ST, but for example on the NXP LPC parts I used, there is a seperate FIFO timeout interrupt which can be used to mop these up.
On the PIC32, there isn't, but you can emulate it by setting up a timer interrupt on exit from the UART handler, to trigger a few hundred microseconds later, and that timer int checks for anything left in the FIFO and forces a UART int if there is.

You can often configure when you get an interrupt, for example 25/50/75% full, the optimum choice depends on how long ( in UART byte times) it takes to get to your handler. In practice this only tends to matter at vary high baudrates ( Mbaud+), where you may set it to int fairly early, to make sure it doesn't overflow by the time you start handling the data. 

And if you use Break conditions ( e.g. for DMX) you really need to look in detail how the fifo deals with these, but that's a while 'nother can of worms...
 

[paulca]

I'm using the STM32 "HAL" libraries for better or for worse.  The nice thing is, as it's MCU land, they just generate all the code into your project.  So you even get the *.s files for setting up the memory mapping registers for flash/sram/etc./etc./ ... and importantly you can edit most of it.

It had more specific callbacks (if statement done for you).  The standard IT hander, which just called the normal "move on" stuff.  But also a half complete and fully complete interrupts.   

There is a real option, maybe requirement to separate out the UART buffers from the protocol they contain.  So instead of scanning for 1 byte in the stream and launching longer read after that, it's more efficient to just fill buffers and use other code to hunt through them.  If I need to go there I can.

[paulca]

On error handling, I tend to have a sort of trade mark in that I write around most of them, make them a "non-issue" or at least, provide them a safe way to protogate by the normal flow without causing anything to become upset.

It's like an Ethernet interface either just dropping a corrupt frame, or forwarding it up to the next layer anyway, with correct length/struct knowing it will get rejected there.

I hate people who over validate input.... then write dumb method code, then validate the return.  Just make the code resilient, make it make sensible choices under odd data.

Nobody like a a noisy attention seeing protocol.

I call it "Meh" error handling.   If ""Meh" ,shrug, don't care", is a good enough answer at any point, it's the correct one.    If there is a way to reach the end of the method correctly and produce a real output, don't be a dick and throw validation exceptions in a library, shut up and do you job and I'll validate he answer....etc. etc...

[mikeselectricstuff]

UART is one of those areas where using HALs is a pretty bad idea, as there are so many different use cases that no HAL is ever going to be an optimal fit. Driving UARTs is simple enough that it's often just as quick to write your own code than learn the API and quirks of someone else's.

[coppice]

UART is one of those areas where using HALs is a pretty bad idea, as there are so many different use cases that no HAL is ever going to be an optimal fit. Driving UARTs is simple enough that it's often just as quick to write your own code than learn the API and quirks of someone else's.

I agree. If you want a really simple case, its really simple to code from scratch. A HAL might make a simple buffered UART easier. However, as soon as you want anything the least bit specialised, which is really common with serial comms between MCUs, you are back to doing it from scratch.

[Psi]

Normally, if you have to do inter-MCU communications, and you have say in how the communications is done, you don't pick UART.   

UART is a very very typical choice for inter-MCU communication, nothing wrong or unusual about that.

UART is often used for inter-MCU coms but it's usually when you are forced to use it for other reasons.
Using a UART forces things on you that are not required or desirable for inter-MCU coms.
The UART standard is intended for data transmission down some wires, not for MCU to MCU coms.

Using SPI is a much more of a convenient method for inter-MCU coms, and much more suited for it.

Using parallel bus is great if you need high speed and want lots of control over everything and want to DMA it.

I2C can be used too, but usually you only when..
 - You have an I2C bus already for some existing peripherals so tapping a slave MCU onto the existing I2C bus is very convenient.
 - You don't need high speed.

[mikeselectricstuff]

Using a UART forces things on you that are not required or desirable for inter-MCU coms.

What things exactly ?

Using SPI is a much more of a convenient method for inter-MCU coms, and much more suited for it.

I'd totally disagree on that one - SPI needs more wires, the master needs to send a byte to receive a byte, much more complicated when more than two devices are involved. Also harder to debug - any PC can sniff a UART link with minimal hardware.

The only advantages I can see are speed, and not needing an accurate clock on either side. Maybe the ability bit-bash it without the timing constraints of doing so with UART.

[Siwastaja]

SPI
...
The only advantages I can see are speed, and not needing an accurate clock on either side.

Hardware packet delimiting is a real advantage, but of course that's assuming one makes use of it (either good peripheral with usable hardware nSS, or some software). With SPI, you can just automagically share two pieces of memory with each other, and the activation of nCS sets the write pointer to the start of the data reliably (unless you fail doing that).

You can of course do the same with UART, add a GPIO pin which resets the receiving state machine. Doing that over the UART protocol itself requires generating and parsing headers, or some form of data manipulation (start/end symbols, escaping, etc.) But it ain't rocket science - if you are fine doing that, and fine with a few megabits per second, UART is more than fine.

[paulca]

Not 100% sure, but I believe the STM32F4 can have input capture interrupts which will start automatically double buffered DMA transfers on UART.  Some of it is orcestrated via the driver code but the actual stuff is done in hardware.

I didn't look into it enough to know if you could combine the two and if "input capture/compare interupts" on UART would be able to trigger on a set byte or word.

I started as simple as I could and it failed, so I went up to interrupts, it looks fairly trivial to go up one more to DMA UART with or without hardware double buffering.  I'll probably not use the input compare stuff at the moment.

For the SPI stuff, it's mostly wrapped in a thin library, but it look like you basically pull on the command line and send the command to write to the screen, then start your DMA transfer for the part of the screen memory you want to draw.  The only thing you need to be careful of, is not clobbering your own DMA transfers or getting stuck in an ISR loop.

[Psi]

The only advantages I can see are speed, and not needing an accurate clock on either side. Maybe the ability bit-bash it without the timing constraints of doing so with UART.

yep, The accurate clock on either side is a big one.

Also
- The extra packet overhead,
- Being forced to packetize it in 5 to 10 bit chucks depending on MCU.
- The latency of the link between the two mcu's is all over the place because of all the start/stop bits, gaps and any hardware fifos
- Your UART interrupt priority is often set in the silicon and so not changeable

.. much more complicated when more than two devices are involved.
..Also harder to debug - any PC can sniff a UART link with minimal hardware.

Fair point


I should probably clarify, when i say inter-MCU comms I mean communication between two MCUs on a single PCB.
When there are multiple PCBs with a cable between them a UART makes more sense.

[mikeselectricstuff]

The only advantages I can see are speed, and not needing an accurate clock on either side. Maybe the ability bit-bash it without the timing constraints of doing so with UART.

yep, The accurate clock on either side is a big one.

Also
- The extra packet overhead,
- Being forced to packetize it in 5 to 10 bit chucks depending on MCU.

.as opposed to 8/16/32 bit on SPI ? Some SPI peripherals allow finer-grained word sizes, but not all

- The latency of the link between the two mcu's is all over the place because of all the start/stop bits, gaps and any hardware fifos

How is this any different from SPI?

- Your UART interrupt priority is often set in the silicon and so not changeable

Very MCU-dependent, e.g. PIC32 allows any priority to be assigned to any int.

.. much more complicated when more than two devices are involved.
..Also harder to debug - any PC can sniff a UART link with minimal hardware.

Fair point


I should probably clarify, when i say inter-MCU comms I mean communication between two MCUs on a single PCB.
When there are multiple PCBs with a cable between them a UART makes more sense.
[/quote]

[coppice]

The only advantages (of SPI) I can see are speed, and not needing an accurate clock on either side.

The clock rate issue can be a big advantage these days, with so many MCUs able to run from an internal, but not very precise, clock.

[paulca]

The only advantages (of SPI) I can see are speed, and not needing an accurate clock on either side.

The clock rate issue can be a big advantage these days, with so many MCUs able to run from an internal, but not very precise, clock.

Yes I found this odd.  The board I have has an internal 8Mhz (HSI) clock which I think generates up to a 48Mhz FCLK.

However the HSE clock is only 8Mhz as well, but the PLL clock can multiply it up to 100Mhz.

I figured it's because the external ceramic resonator has a high quality clock and can be multiplied up more?

[Psi]

.as opposed to 8/16/32 bit on SPI ? Some SPI peripherals allow finer-grained word sizes, but not all

32bit is much more useful

- The latency of the link between the two mcu's is all over the place because of all the start/stop bits, gaps and any hardware fifos

How is this any different from SPI?

Sorry Mike, Think i used the word latency incorrectly.
I mean SPI can be used for a continuous stream of bits when needed.
An example is using SPI output pin to write an On Screen Display line by line over the top of a composite video signal using SPI DMA.  Definitely not a common use case but a good example of when you need a stream of bit data without interruptions for start/stop/gap bits and with consistent timing. 

- Your UART interrupt priority is often set in the silicon and so not changeable

Very MCU-dependent, e.g. PIC32 allows any priority to be assigned to any int.

Yeah, fair point.

[mikeselectricstuff]

I mean SPI can be used for a continuous stream of bits when needed.
An example is using SPI output pin to write an On Screen Display line by line over the top of a composite video signal using SPI DMA.  Definitely not a common use case but a good example of when you need a stream of bit data without interruptions for start/stop/gap bits and with consistent timing. 

Yes, the ability to generate an arbitary bitstream can be useful, I once abused an SPI port to generate DMX ( that was on an NXP part that allowed 10 bit SPI)

[Siwastaja]

Setting SPI word size to 16 bits, for example, makes DMA operations over 16-bit variables (aligned to 2 bytes) atomic. The whole shebang can be this easy:

struct
{
    int16_t measured_current;
    uint16_t measured voltage;
} info;

struct
{
    int16_t current_setpoint;
    uint16_t voltage_setpoint;   
} command;

TX_DMA_STREAM->MEMORY_ADDR = &info;
TX_DMA_STREAM->N_DATA = sizeof info / sizeof (uint16_t);

RX_DMA_STREAM->MEMORY_ADDR = &command;
RX_DMA_STREAM->N_DATA = sizeof command / sizeof (uint16_t);

// circular mode on
// keep triggering the DMA
// just read and use command, and write info
// measured_current and measured_voltage might not be from the same packet, but often this does not matter