STM 32F4 stack overflow protection

[peter-h]

The project runs FreeRTOS, with the #4 memory model, where a fixed buffer is declared and the RTOS allocates process stacks out of that.

That buffer is 48k and goes into the CCM.

The stack is 16k and takes up the rest of the CCM, obviously working down from its far end. This is used in main.c (very little there), for ISRs, and some other stuff, none of it big.

I realise that in an embedded system there is little you can do if things run away but is there any way to set up a trap for SP going below some address?

Probably more useful would be to zero the whole CCM and have a task running every say 10 secs which looks for nonzero values and prints out some sort of graphic representing these... and run the target for a few days. With a VT100 terminal one could do a nice graphic

[MadScientist]

Depending on the exact M4 core you could use the memory protection unit to provide a no access zone between the two memory zones , the region size on M4 can be quite small so only a small amount of ram is wasted.

[peter-h]

It is a 417. I will dig around - thanks.

This is probably asking way too much but is there any way to use the Cube IDE debugging tools to generate such a graphic? It seems to have 100x more features than anybody could ever use Probably that would need the use of temp breakpoints, like any other form of execution profiling, which is another thing that would be interesting.

[Alti]

If this memory is allocated by OS then this is portion of memory is called heap. It is the code execution flow that decides the allocation, size, sequence, freeing etc. Of course you could play with MPU and run-time configure it but that is going to be quite an assignment.
Reconsider your decision about dynamic (run-time) memory allocation, most uC embedded applications do not require heap. Then memory partition is made statically at linking stage (use linker for that). Also, when all stacks have fixed addresses, MPU configuration becomes super simple.

As for using DWT for tracing memory problems - sure, you could do that but there is quite a limited count of those data traps (I think only 2 in CM3/4).

I do not know Cube.

[cv007]

Put the stack at the bottom of ccm, when it goes below a usable address- bus/some fault.

[lucazader]

As pointed out by cv007 you can flip the standard stack location

You can see it explained a bit more in this rust code example (mainly just the linker section layout is the important part.)
https://github.com/knurling-rs/flip-link

Basically instead of allocating the stack at the top of ram, and letting it grow down. Towards your heap/const variables or other important things, running into issues if the stack usage grows to be larger than anticipated.
All you do is allocate the stack at a mid point in ram, and let it grow down, as soon as it goes below the lowest valid memory address this will cause some sort of bus/system or hardfault which you can then detect etc.

This is explained clearer, in more detail and with diagrams in the github link.

Hope this helps

[peter-h]

That's a cunning trick - many thanks

BTW the heap is in main RAM. I have the usual setup there i.e. data, bss, and then the heap can grow all the way to the end of the RAM via the limit imposed in _sbrk.

The more basic problem is that a trap is of little use in an embedded system. I will probably have an RTOS task running say once a minute which scans the whole 64k for nonzero locations, so one can see actual usage. Although that isn't perfect since you could call a function which allocates
uint8_t fred[10000];
and suddenly the SP is 10k further down and if that buffer doesn't get used, the memory will not change from zero.

[newbrain]

I will probably have an RTOS task running say once a minute which scans the whole 64k for nonzero locations, so one can see actual usage. Although that isn't perfect since you could call a function which allocates
uint8_t fred[10000];
and suddenly the SP is 10k further down and if that buffer doesn't get used, the memory will not change from zero.

Pay attention to the fact that if you using one of the common RTOSes, the stack for each task is held in that task's allocated memory.

E.g. for  FreeRTOS, in the heap* unless you explicitly enable static allocation. For Azure RTOS/ThreadX it's more or less the opposite.

Your (auto) uint8_t fred[10000] inside a task is almost guaranteed to pee out of the toilet, unless you provided enough space at task creation time.

* As a further reminder: FreeRTOS.heap has in general nothing to do with the C library one, from some post above I had the impression you were mixing them up a bit(apologies if I'm wrong)

[peter-h]

Yes; FreeRTOS is given this

CCMRAM static uint8_t ucHeap[ configTOTAL_HEAP_SIZE ];

where the size is 48k. It then allocates process stacks out of that.

We do actually have a problem in this area, in that if in

xTaskCreate(vSPI3Task, (portCHAR *) "SPI", configMINIMAL_STACK_SIZE, NULL, tskIDLE_PRIORITY, NULL);

where the stack size is defined as  ((uint16_t)512), and this works, but if say the stack is increased e.g.

xTaskCreate(vSPI3Task, (portCHAR *) "SPI", configMINIMAL_STACK_SIZE*5, NULL, tskIDLE_PRIORITY, NULL);

then the whole thing more or less hangs, but it is not clear why it should hang with 2.5k of stack on that process, when there are only a few processes on it, and the whole lot can go up to 48k. This will be one of the things to track down, and it may take a long time.

My comment about the 10k buffer on stack was about some code not running under the RTOS.

You are quite right about me mixing things up; I was told by the other guy who worked on this that the 48k RTOS buffer came out of the general heap, but this was not the case, and anyway would seem to be pointless (why malloc x k and then give that to another bit of code which uses its private version of malloc to allocate small pools out of that?). It just so happened that both this buffer and the general heap were 48k, which confused the hell out of me trying to track down the huge BSS usage

Currently the RTOS buffer is in CCM memory and the stack is in the remaining 16k of CCM, and it would be good to detect if the stack goes anywhere near its bottom limit.

[SiliconWizard]

Sorry for the silly question, but have you asked FreeRTOS's team? I'm not familiar with FreeRTOS's code at all, but FreeRTOS developers can probably answer your question much better than we can.

[newbrain]

the stack size is defined as  ((uint16_t)512), and this works, but if say the stack is increased e.g.

xTaskCreate(vSPI3Task, (portCHAR *) "SPI", configMINIMAL_STACK_SIZE*5, NULL, tskIDLE_PRIORITY, NULL);

Remember also that FreeRTOS stack sizes are in words, not in bytes.
Very few tasks with that size would be enough to go out of memory (512 × 4 × 5 = 10240).

Did you check the return values of xTaskCreate()?

EtA: Are you aware that FreeRTOS supports stack overflow checking via a configuration flag?
 It of course (slightly) affects performance, but could be useful in a debug phase.

[Silenos]

You can catch task stack corruption by doing checksums/hashes of the task frame during context switch. Like this what Newbrain posted. If you want to somehow recover from such an event, you can get creative with implementing the idea of c++/java exception mechanism on application architectures - yes, more overhead; you don't get such features for free on cortex-m architectures.

My comment, correct me if my assumptions on your project are totally wrong.
From my experience:
On such architecture dynamic allocations of memory or, mygod, tasks are evil. It just opens the gate for unpredictable outside world events to overflow your system with allocations on nonvirtualised memory space. Totally untestable, so unreliable. Just no.

If you need dynamic allocations/deallocations, like those SPI transactions of yours, just queue them in static blocking FIFO, inside one static task. If you can't tell the max size of a FIFO, or you can't handle FIFO saturation on application layer, then the system design is flawed. You can't conjure more space for more tasks/transactions up out from nowhere anyway.

If you handle things like eg. Ethernet: you have a chance to be burst flooded with big packets at max bandwitch. They will fill your buffers up in no time, assuming you cannot process them in time.
I have dealt with such scenario this way: declare the static buffer table, usually the size of remaining ram (or tables on multiple ram sections) and run a custom allocator over those tables. So yes, this basically functions as heap+malloc or w/e allocation your RTOS uses.
Though my allocator was made specifically for one-way linked list, dealing with spanning over hardware ram sections, riding on DMA, handling fragmentation and having an specific algorithm to select the the incoming or already stored data to be dropped/overwritten in case of saturation. Nothing spilled outside of those buffers to this day.
Though that's prolly an overkill on your F4, imo the "standard" heap for such a buffer would be enough, assumming you implement saturation scenario atop the allocator calls. Using heap for random buffers and variables? No.

Trying to reinvent your RTOS to be safe from "this guy allocated char[10000] and crashed the thing" - "how can I prevent this?" is so wrong.
It is a waste of time doing that on embedded system fitting on F4. Go, exert domination upon that guy and make him not to allocate more than 500 bytes or w/e - it is much easier.
You are not working on Windows or sth made with purpose to be resistant against flawed/broken/malicious tasks, you don't need all that project-irrelevant crust, you need real time OS to control your thing with tasks expected to be correct, as you have control over those tasks' code. Gather your coworkers, print them "task design guidelines and constraints" like max allocation space or max stack depth and do the actual job.

[peter-h]

"have you asked FreeRTOS's team"

No; one tends to assume that there is no support for anything in hardware or software these days Certainly that is the case for anything to do with ST. Their forum is full of questions and with mostly no answers. And same all over the internet.

"Remember also that FreeRTOS stack sizes are in words, not in bytes."

OK that would be 512 32-bit words; 2k bytes. Learn something every day

One of the functions called in the RTOS tasks allocates a 2k buffer, so that is probably crapping over something else.

"Very few tasks with that size would be enough to go out of memory "

The tasks currently running are allocating (words)

1024
1024
512
512

which totals about 12k bytes.

"Did you check the return values of xTaskCreate()?"

No

"Are you aware that FreeRTOS supports stack overflow checking via a configuration flag?"

Yes; currently disabled. I had a dig around that code when it was crashing when its buffer was in main RAM (0x2...) while the entry stack (which remains used for ISRs) was in CCM (0x1...) and I suspected this was causing a crash.

The news that the RTOS allocation is in words not bytes explains why it is working, and may explain why it crashes when I allocate 512*5 to one of the tasks. But that still doesn't add up to 48k.

Silenos - I fully understand your drift, and I don't like the idea of a heap at all. With decades of asm programming one learns to keep stuff deterministic even if it wastes RAM. I like static buffers and just re-use them.  Messy but very reliable if you are organised, but nobody does that in C. But the ethernet stuff is above my pay grade and someone else is doing that. AIUI the ST ethernet code uses ~10 1500-byte (MTU-sized?) statically allocated buffers and drives these with DMA. They are not malloced. The stuff which is likely to be malloced is buffers for doing certificate handling; I had a dig around and some of these are 16k+, so it may be hard to go fully static on that, but obviously one must follow every malloc with a matching free otherwise you get fragmentation and the whole thing will fall apart. The guy doing this knows that.

What is interesting is that when I moved the 48k RTOS buffer and the stack into CCM, the ethernet stuff stopped working. No DHCP even. I wonder if there is a buffer on the stack? CCM is not DMA accessible.

Nobody is going to be allocating char[10000] inside a function because it would be stupid and there are only 2 of us working on this, but one can get funny stuff happening and a stack can grow out of control and this can be undiscovered for ages, in the right circumstances, hence my desire to see how far down the stack is growing.

This is a great learning experience for me - thank you all!

[newbrain]

On such architecture dynamic allocations of memory or, mygod, tasks are evil.

I totally agree, in part 

For FreeRTOS, if task, queues, semaphores etc. are allocated once at startup, and are not ever deleted, it's just a simpler way to write your software.
The memory is dynamic in name only, and the APIs are simpler.

Of course, in some environment, dynamic allocation is simply forbidden in any form - so the static memory version of the creation APIs must be used.

Other RTOS, as I mentioned above, take a different approach, e.g. ThreadX only has APIs equivalent to the static version of the FreeRTOS ones, the thread type is not an opaque handle but an actual structure:

Code: [Select]

TX_THREAD  thread_0;At thread creation, both the thread struct and the stack area are passed as a pointers, which can be either allocated statically (as shown above for a thread) or carved out from a memory pool:

Code: [Select]

    tx_byte_allocate(&byte_pool_0, (VOID **) &pointer, STACK_SIZE, TX_NO_WAIT); /* Stack allocated from pool, thread statically allocated */
    tx_thread_create(&thread_0, "thread 0", thread_0_entry, 0, pointer, STACK_SIZE, PRIO, PREEMPT_THR, TX_NO_TIME_SLICE, TX_AUTO_START);


[SiliconWizard]

"have you asked FreeRTOS's team"

No; one tends to assume that there is no support for anything in hardware or software these days

And this is very wrong. While more people "tend to assume there is no support", companies in turn just assume support is not needed, and will cut costs on that. This leads to a bad situation. We need to make companies f*cking responsible for what they sell and for the associated required support. Not asking won't help! It lets them get away with it.

Certainly that is the case for anything to do with ST. Their forum is full of questions and with mostly no answers. And same all over the internet.

Maybe, but if people buying ST stuff are OK with that (meaning: they still buy), then ST is not likely to improve the situation. See above.

But anyway, I was not suggesting asking questions to ST, their forums or any other related forum. Your problem is FreeRTOS-related, and as I suggested, it makes a lot more sense to ask for support from FreeRTOS, not from ST.

While it's open-source, they have a number of commercial partners behind them and get significant funding. So you're certainly "entitled" to some support. Open source doesn't necessarily mean that you're on your own.

Taking a look at their forums, they seem reasonably active, had you tried them?
https://forums.freertos.org/

[aandrew]

The 40x/41x have an MPU; just set the last stack location as not-accessible and catch the fault.

[peter-h]

"We need to make companies f*cking responsible for what they sell and for the associated required support. Not asking won't help! It lets them get away with it."

I agree, but that bus left many years ago. In the 1980s, my then company had a direct account with Zilog (we used to buy lots of their parts and eventually they agreed to bypass the distis) and we had great support from them. By 2000, the whole industry got rid of employing anybody "technical". So now if you have a problem, you google on it, you find hundreds or thousands of hits on exactly the same problem, and after hours of reading these (most of them will be dead ends, with the few "solutions" not working) you may find something which fixes the problem. In turn, since firms like ST are more or less totally disconnected from the users, bugs take years to fix. For example the Cube IDE bug which generated a link script with the main RAM as 192k (when actually it is 128k RAM, with the 64k CCM non-contiguous) took ~ 3 years to fix, but how many 32F4 users would have independently discovered that? ST don't have a contact email / web form for issue reporting, so nobody bothers, because everybody knows nobody will respond.

The FreeRTOS forum looks quite good. Some intelligent replies there Thanks for the tip. Unfortunately my C needs to get a lot better.

[newbrain]

ST don't have a contact email / web form for issue reporting

Edit: checked it's still there, easily accessible from the home page menu.
Cannot check that now, but:
They had a support page about three years ago, and now issues on the Cube HAL can also be reported through their GitHub repo.
Are they responsive?
Well, they answer in a reasonable time. Then it takes ages and a number of releases to see the solution (dutifully provided with the ticket) in place...
But, after all, I'm just a hobbyist, not a significant customer in any way.

[peter-h]

Thanks - that's useful to know. I have a login - it is the same site as their forum, on which almost nobody answers (properly).

They would never know if you are a customer anyway, because all but the biggest users buy via distis, so the mfg has no idea who is buying what.

[newbrain]

Thanks - that's useful to know. I have a login - it is the same site as their forum, on which almost nobody answers (properly).

They would never know if you are a customer anyway, because all but the biggest users buy via distis, so the mfg has no idea who is buying what.

In the form, IIRC, they have fields for the project stats, like development phase volume and probably reference FAE.
One can lie, but it's easy for them to find out...

[peter-h]

Turns out that despite having a login to the ST site I don't have access to creating tickets, and need to ask my "community manager" to get that, so who is going to bother?

On the original topic, I will try to write some code which dumps the CCM as "character graphics" to a VT100 terminal, slowly. It would also be easy to have an RTOS task which examines some stack locations for nonzero values and warns, or increments a counter. One could do the same in the IDE, with breakpoints with a counter, on these locations, to see how far the stack has grown, without stopping the system.

It is possible that this is where the expensive Stegger debuggers (an early thread of mine, inconclusive as to what they do that worth having) might have an advantage, because they claim to run a lot faster.

[newbrain]

Turns out that despite having a login to the ST site I don't have access to creating tickets

I'm puzzled, I never had to ask for any special permissions or contact anyone.
But things might have changed since I registered.

In the picture, the form I get clicking on Support, then "Support Request - via web form".

[peter-h]

That's funny. I emailed them about this. The reply was "Thanks to please confirm is this related to manufacturing domain" to which i replied I am just an old engineer, not  familiar with modern corporate lingo

Today, it works and the above form comes up