/rant.

[Siwastaja]

I tend to take a different approach.  Especially as I started with the auto-generator and HAL.

Your mistake is the same as with so many others - they assume ST's auto-generator and ST HAL are significantly helpful. They are not.

This being said, take something which should be easier: higher level, general purpose OS designed for userland applications, much more tested, much more widely used: write UART TX/RX on linux, using the standard POSIX interface (and possibly linux-specific system calls). Surprise surprise: it's not any easier; you will spend hours when you do it for the first time. There is no simple
init_uart("/dev/ttyS0", 115200"); uart_send(buf, sizeof buf); type stuff, you have to deal with weird interfaces and even more weird legacy things.


With others (ST, etc.) being human and prone to poor design, laziness and mistakes, and with yourself being human and prone to the same issues, it's usually best to follow NorthGuy's strategy. Once you feel confident, you can write larger parts at once.

[paulca]

I got it syncing without DMA.

Idle time on the Tx.
ReceiveToIdle with a longer than required buffer.
Detect successful idle to idle by confirming you got 2xStatistics_t size read and not the full buffer.

Only then did the buffer begin at byte 1 of the struct and could be mapped to the struct and accessed.  Only then did the FRAME_SYNC word end up correct.  Only then did it render a far too fast to read negative dbm value

Next steps are to work out how to get that trick to work with DMA.  Then torture test it a bit.

[paulca]

Grrr.

I got DMA working, but with about 30,000 errors per second and only about 1 or 2 successes.

The issue seems to be the interaction with the UART interrupts and DMA.  If you start the DMA receive in the middle of the stream, it immediately errors out.  It starts the DMA stream in "receive to idle" mode.  However, if any error flags are set on the UART when the DMA attempts to start it will set the UART back to "Standard" mode.  This is then validated in the HAL call and aborts returning HAL_ERROR;

Spinning on it is literally the only way to get it to start while it's idle and only by luck.  The first bit it receives, if it decides it's not a start condition, it flags it as an error, Frame error for example which aborts the process.

I have a few more things to try before I either write my way around that HAL function or go back to circular and picking the data out by hand.

I believe it's a timing issue with the HAL code.  It starts the UART and immediately starts the DMA, then it goes and checks if everything started corrected.  No surprises that, no, it didn't.  Because the UART started in the middle of an existing stream and the very first thing it found was garbage, so it raised a concoction of errors.  Overrun, frame error, etc. etc.  I figure if I enable the UART myself, let it spit its errors out, clear the flags and then start the DMA on receive to idle while the UART flags are clear.

Why DMA?  It's a UI it's two main processes are both long winded transfers.  UARTs and TFT SPI.  At the same time it has to find time for the encoder polling (later) and also sending async on the UART.  Doing all of those in sequence would drop the "frame rate" to about 0.5fps and make the encoders horrible.

[langwadt]

Grrr.

I got DMA working, but with about 30,000 errors per second and only about 1 or 2 successes.

The issue seems to be the interaction with the UART interrupts and DMA.  If you start the DMA receive in the middle of the stream, it immediately errors out.  It starts the DMA stream in "receive to idle" mode.  However, if any error flags are set on the UART when the DMA attempts to start it will set the UART back to "Standard" mode.  This is then validated in the HAL call and aborts returning HAL_ERROR;

Spinning on it is literally the only way to get it to start while it's idle and only by luck.  The first bit it receives, if it decides it's not a start condition, it flags it as an error, Frame error for example which aborts the process.

I have a few more things to try before I either write my way around that HAL function or go back to circular and picking the data out by hand.

I believe it's a timing issue with the HAL code.  It starts the UART and immediately starts the DMA, then it goes and checks if everything started corrected.  No surprises that, no, it didn't.  Because the UART started in the middle of an existing stream and the very first thing it found was garbage, so it raised a concoction of errors.  Overrun, frame error, etc. etc.  I figure if I enable the UART myself, let it spit its errors out, clear the flags and then start the DMA on receive to idle while the UART flags are clear.

Why DMA?  It's a UI it's two main processes are both long winded transfers.  UARTs and TFT SPI.  At the same time it has to find time for the encoder polling (later) and also sending async on the UART.  Doing all of those in sequence would drop the "frame rate" to about 0.5fps and make the encoders horrible.

why not skip DMA and just use the RXF interrupt?

[DavidAlfa]

Peripheral ISRs are normally not required in DMA mode.
You only care about the DMA, waiting to the transfer complete flag.

All that HAL rant seems from people who actually didn't spend enough time with it.
You can do most peripheral functionalities with just few calls.
Now, you must first peek all the HAL functions and figure out what each one does, because the docs are anything close to great.

Edit: For some reason HAL UART DMA needs both ISRs (DMA and UART) enabled.
Anyways, this is working for me.

Code: [Select]

void uart_dma_test(void){   // Called from the main loop

    static uint32_t tim;
    const char msg[] = "This is a very long message sent using DMA!\r\n";

    if(tim <= HAL_GetTick()){
        tim = HAL_GetTick()+1000;
        HAL_UART_Transmit_DMA(&huart1, (uint8_t *)msg, sizeof(msg));
    }
}

So I wonder if you're going too fast, not properly checking if the DMA is done?
Just check the HAL return. If getting HAL_BUSY, skip, but don't threat is as error.
This works non-stop:

Code: [Select]

void uart_dma_test(void){
    static uint32_t tim;
    const char msg[] = "This is a very long message sent using DMA!\r\n";

    uint8_t res = HAL_UART_Transmit_DMA(&huart1, (uint8_t *)msg, sizeof(msg));
    if(res == HAL_OK || res == HAL_BUSY)
        return;
    else{
        uint8_t blink;
        if(res == HAL_ERROR)
            blink=50;
        else                // HAL_TIMEOUT
            blink=200;
        while(1){
            if(tim <= HAL_GetTick()){
                tim = HAL_GetTick()+blink;
                HAL_GPIO_TogglePin(LED_GPIO_Port, LED_Pin);
            }
        }
    }
}

[paulca]

Do catch up.  The Tx has been working since Wednesday night!

Now, run the Tx in a loop and try and receive that. On another STM32 with DMA.

[DavidAlfa]

Oh it's the RX then? Sorry, TL;DR.

Probably the RX device can't hold up, as it needs more time to process the data after receiving?
How about inserting a 10ms delay after each TX DMA transfer, to see if it fixes the issue, then reducing until finding the magic number?

[paulca]

Oh it's the RX then? Sorry, TL;DR.

Probably the RX device can't hold up, as it needs more time to process the data after receiving?
How about inserting a 10ms delay after each TX DMA transfer, to see if it fixes the issue, then reducing until finding the magic number?

How does it know when to start and how does it know when to stop?  The stream is running already.

If you try and start a DMA transfer with HAL the UART throws a frame error or other error as it picks up in the middle of the stream in the middle of a byte.  That then aborts the DMA.

So I have inserted some idle time (10ms) between transmissions.  I am currently finding out how to start the UART.  Let it settle, clear it's flags and then ... wait on it's idle flag (or interrupt) and immediately start the DMA while the UART is "idle".

[cv007]

(Oh... I want birectional later, but shit this uni-d should be easy?).

If you will be using another pin later, use it now as a way to sync and when you go bidirectional later the syncing can be done via communications.

[langwadt]

Oh it's the RX then? Sorry, TL;DR.

Probably the RX device can't hold up, as it needs more time to process the data after receiving?
How about inserting a 10ms delay after each TX DMA transfer, to see if it fixes the issue, then reducing until finding the magic number?

How does it know when to start and how does it know when to stop?  The stream is running already.

If you try and start a DMA transfer with HAL the UART throws a frame error or other error as it picks up in the middle of the stream in the middle of a byte.  That then aborts the DMA.

So I have inserted some idle time (10ms) between transmissions.  I am currently finding out how to start the UART.  Let it settle, clear it's flags and then ... wait on it's idle flag (or interrupt) and immediately start the DMA while the UART is "idle".

process the stream byte by byte using the RXF interrupt until you have found your a valid frame header, then start the DMA

[paulca]

process the stream byte by byte using the RXF interrupt until you have found your a valid frame header, then start the DMA

At one point I did have this in place.  It wasn't working.  However.  That may have been a reflection of the unpacked struct over-running the buffer I had used and thus there were no consistent "FRAME_SYNC" bytes.

I have considered this and other approaches, but the one I ended up with (not yet successful), is to start the UART, start a timeout.  Spin BUSY=false and IDLE=true flags, then start the DMA.

Not got that working yet.  However the condition does readily exist in sane quantities (if that makes sense), it's not a sparse or rare event.  Suggesting it might a good avenue.

[AVI-crak]

process the stream byte by byte using the RXF interrupt until you have found your a valid frame header, then start the DMA

In 9-bit mode, the search for the start word (address) occurs at the hardware level. In 8-bit mode, you can hardware search for a start character, but only in text-receiving mode.
In raw data receive mode, you will have to process each byte in software, because hardware tracking cannot be disabled in receive mode, and the UART itself will not have time to enter receive mode after an overload, and is guaranteed to lose data.

paulca - there are many ready-made code examples on the net, completely free. It is not necessary to reinvent the wheel without having basic knowledge - it is always bad.

[paulca]

paulca - there are many ready-made code examples on the net, completely free. It is not necessary to reinvent the wheel without having basic knowledge - it is always bad.

Provide one example.

[paulca]

Look I get it.  This SHOULD be simple.

Why is it not?  That's the point of the thread and it's title.

[langwadt]

process the stream byte by byte using the RXF interrupt until you have found your a valid frame header, then start the DMA

In 9-bit mode, the search for the start word (address) occurs at the hardware level. In 8-bit mode, you can hardware search for a start character, but only in text-receiving mode.
In raw data receive mode, you will have to process each byte in software, because hardware tracking cannot be disabled in receive mode, and the UART itself will not have time to enter receive mode after an overload, and is guaranteed to lose data.

none of that makes a difference if you start receiving in the middle of a stream and no matter what you'll need the resources to process the data

[MarginallyStable]

I have seen many attempts by users to pass raw binary data over the UART, all of which failed.
UART is for text.

Haven't been around long have you.

[langwadt]

Look I get it.  This SHOULD be simple.

Why is it not?  That's the point of the thread and it's title.

how could it be made simpler? if tx and rx starts a random times you can be off by bits or bytes so you have to devise a way to get the ends in sync

[paulca]

Look I get it.  This SHOULD be simple.

Why is it not?  That's the point of the thread and it's title.

how could it be made simpler? if tx and rx starts a random times you can be off by bits or bytes so you have to devise a way to get the ends in sync

My very first implementation of this I did, char by char interrupt, read the stream until receiving the last byte in the struct "0xF8", start the DMA stream in circular mode.

[AVI-crak]

Look I get it.  This SHOULD be simple.

Why is it not?  That's the point of the thread and it's title.

Yes, much easier, you just need to do it differently.
Let's say you have a structure that needs to be edited. Structure with raw data: double precision numbers, + unsigned numbers, + small bit fields, in general, complete inconsistency. It doesn't matter what's inside, the main thing is that the first in the structure should be a unique symbol. The characters "0-9,A-F" are already taken, zero cannot be transmitted, "\n,\r" - there is no point in transmitting (short distance). There are a lot of characters left for the role of the starter, let's say "G".
You have a data sending event not in itself, but a very specific result of the program execution, when all the data in the structure becomes relevant and ready to be transferred. That's when it's ready, then you can call the function for forming a string from the structure into one of the buffers (there should be two). There, if the first buffer is already involved in the transfer, then you need to switch to the second one. When forming a string, you can discard the standard "0x" (short distance). After collecting the line, we check the operation of DMA, and if it is busy, then we wait, then everything is standard.

Now we need to receive the data.
DMA is configured for a ring buffer, and works to receive without stopping. The UART has a character match interrupt in which you just need to read the DMA position, and write (head) so that it is visible to your processing functions. The processing function itself is called either through a timer or non-stop. As soon as a number other than 0xFFFF appears, it must be read into a new variable, and then replaced with 0xFFFF.
Next, we rewrite the contents of the ring buffer into the temporary buffer in the waiting loop for a new value of the DMA counter (read directly from the register). Or use a delay (the size and transmission time are known). You can restore binary data on the fly, or you can when a sufficient amount has already accumulated, it doesn’t matter.
Data checksum verification and recognition of different structures - at your discretion. Once everything fits together, you can copy the buffer data into the link structure for further use.

What the Internet does not have is the functions to handle the transmission failure mode. For the physical layer, it seems to be as it is, but write something purely into the debug port.

[paulca]

I did spot the character match interrupt while debugging.  Interesting feature.

There isn't really much in the project so I it's total disposable.... well on the Rx side.  The Tx is very busy.  So busy that if I want to run it in debug without optimisations I have to disable parts of the processing with an ifdef. 

You are suggesting not to use idle time to detect the start of the data, but to just loop it in a circular/double buffer on both sides and use the character match interrupt to point to the start of the data.

That has some sharp edges too though.  Knowing where the start of the data is doesn't tell you if the full set is received.  I suppose it's just a state machine.  DMA gives me something, I copy it from the "head" last pointed to by the character match interrupt until I reach the end of that section of ring buffer.  If I find another delimiter/start char along the way I can mark that buffer as ready, if I don't, I go back to waiting on more.

The only mechanic in that I don't like is the copy.  Although, a memcopy of a 100 or so bytes isn't going to bother anything and makes the "shared buffer" problems a lot more simple when you can have public/private *copies* rather than pointers.

[DavidAlfa]

Code: [Select]

typedef struct Statistics_struct {
float rms[8];
float peak[8];
char FRAME_SYNC;
} Statistics_t;
Statistics_t stats[2];

Statistics_t *busy_stats = &stats[0];
Statistics_t *available_stats = &stats[1];


Code: [Select]

HAL_UART_Transmit_DMA(&huart2, (uint8_t*)&stats[0], sizeof(Statistics_t) * 2);


Code: [Select]

void HAL_UART_TxHalfCpltCallback(UART_HandleTypeDef *huart) {
if (huart == &huart2) {
busy_stats = &stats[1];
available_stats = &stats[0];
return;
}
}
void HAL_UART_TxCpltCallback(UART_HandleTypeDef *huart) {
if (huart == &huart2) {
busy_stats = &stats[0];
available_stats = &stats[1];
return;
}
Debug_UART_TxCpltCallback(huart);
}
void HAL_UART_ErrorCallback(UART_HandleTypeDef *huart) {
Debug_UART_ErrorCallback(huart);

}
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart) {
Debug_UART_RxCpltCallback(huart);
}


Code: [Select]

bool handled = Analysis_Handle();
if (handled) {
haveHandled++;
}
if (haveHandled>10) {
haveHandled=0;
     // snip LED indicator code
available_stats->rms[0] = *output1L.active_rms;
available_stats->rms[1] = *output1R.active_rms;
available_stats->rms[2] = *output2L.active_rms;
available_stats->rms[3] = *output2R.active_rms;
available_stats->peak[0] = *output1L.active_peak;
available_stats->peak[1] = *output1R.active_peak;
available_stats->peak[2] = *output2L.active_peak;
available_stats->peak[3] = *output2R.active_peak;
}


So where are your statistics variables, modified inside the ISR callback, declared as volatile?

[Siwastaja]

My favorite pattern lately:
* no DMA
* A lean, simple peripheral interrupt which parses just the stream delimiters (no content parsing), writing to one of the two buffers,
* when said ISR detects full packet condition, it
  * swaps the two buffer pointers,
  * Triggers lower-priority software interrupt, which will run after the ISR returns (on Cortex-M, NVIC->STIR)
* Software-triggered packet handler ISR processes the buffer (no memory copying), it has time until the next full buffer finishes thanks to double buffering.


Of course, even a colossally simple ISR will be at least some 50 clock cycles including interrupt latency. Maybe if you set the UART in some highest possible baud rates, it might be too much.

[DavidAlfa]

On every UART DMA transfer, three ISR are happening:
- DMA HT
- DMA TC
- UART TC

HAL enables all DMA interrupts by default:

Code: [Select]

HAL_StatusTypeDef HAL_UART_Transmit_DMA(UART_HandleTypeDef *huart, uint8_t *pData, uint16_t Size)
    ...
    /* Set the UART DMA transfer complete callback */
    huart->hdmatx->XferCpltCallback = UART_DMATransmitCplt;

    /* Set the UART DMA Half transfer complete callback */
    huart->hdmatx->XferHalfCpltCallback = UART_DMATxHalfCplt;

But UART_DMATxHalfCplt is empty by default, does nothing, made so the user can override this weak function for its own purposes, thus wasting resources when unused.

You can avoid this by either modifying the HAL code:

Code: [Select]

huart->hdmatx->XferHalfCpltCallback = NULL;
Or by inmediately disabling the HT interrupt after calling the HAL txfer:

Code: [Select]

__HAL_DMA_DISABLE_IT(huart1.hdmatx, DMA_IT_HT);

[AVI-crak]

The only mechanic in that I don't like is the copy.

It is possible to avoid copying from the ring buffer, for this it should have a size from 5X to 20X the size of the structure (data line).
A small clarification - text is transmitted through the physical interface without a terminating zero, exactly twice the size of the structure, literally. It can be observed in the terminal with the automatic line forwarding mode ("\n,\r" do not pass!!) - without character distortion.
In fact, the assembly of the string and subsequent processing of the string is as fast as possible, one byte requires 14 assembler instructions.
And yet, the memory area of ring buffers and buffers of transmission lines - you need to locate in a memory area isolated from caching. Otherwise, debugging will be a lot of fun.

[betocool]

If I may, @paulca, I think a better approach would be to use Cube MX to initialise all the peripherals, except DMA. As someone before mentioned, HAL DMA is clunky at best and confusing.

I find it way easier to init UART, and then manually init the DMA channel for TX and RX. Admittedly, RX can be a bit trickier because if you don't know how many bytes you're expecting to receive, but it's still doable. You'd be surprised how easy it is to configure DMA transfers using registers and DMA half and complete interrupts.

Another suggestion... I've never had any problems using binary protocols over UART. I use protocol buffers (protobufs on the PC end and nanopb on the micro end) to transfer what is effectively a struct packaged in another format.

Believe me, it is a "fail fast" approach, and easy enough to make changes without looking at the HAL function, however, you do need to have a look at the DMA configuration. Once you get that under your belt, you'll see how easy it is to implement for other peripherals as well.

Cheers,

Alberto