FreeRTOS: a large chunk of RAM used up at the base of the allocated RAM

[peter-h]

I have 64k allocated, using heap_4.

I've already posted about stack allocation here
https://www.eevblog.com/forum/microcontrollers/weird-issue-with-local-variables-located-*outside*-a-freertos-task-stack/msg4181401/#msg4181401
and that seems well sorted now.

However I have another issue, which is using up a lot of RAM - almost 16k. At the base of the 64k block, address 0x10000000 onwards, I see the various debug statements laid out



They are spaced out at 176 byte intervals which is a huge waste of space, but also many are duplicated several times when they appear only once in the code.

Googling on FreeRTOS memory allocation doesn't illuminate this.

I realise any test strings defined within an RTOS task will be allocated on that task's stack - same as a string defined inside any other function. But what's happening here?

Or it could be some buffer, which would explain the repetition, but what is it? No such buffer is defined anywhere. The whole 64k block is dedicated to the RTOS, with

Code: [Select]

#define configTOTAL_HEAP_SIZE                    ((size_t)64*1024)

#define CCMRAM __attribute__((section(".ccmram")))

// RTOS buffer, used for process stacks etc.
CCMRAM uint8_t ucHeap[configTOTAL_HEAP_SIZE];

After around 0x10003918 I start to see the expected task stacks, which were prefilled with 0xA5 by the RTOS.

The program runs and this has come to light only now, when I am looking for more task memory.

[peter-h]

Well, the answer turns out to be quite simple. This area is where FreeRTOS puts its message queues.

But, despite the reams of doc online on the RTOS, even now I can't see a "memory map" anywhere. The culprit was this statement

Code: [Select]

    g_usb_MsgQueue = osMessageQueueNew(MSGQUEUE_OBJECTS, sizeof(MSGQUEUE_OBJ_t), NULL);
    if (g_usb_MsgQueue == NULL) {
       ; // Message Queue object not created, handle failure
    }
which for some reason had MSGQUEUE_OBJECTS*5. Some fossil code. The reason the debug statements were seen in there was that this RTOS feature is used for a USB VCP queue. We aren't actually using the message queue feature for anything. I tend to use global booleans etc as flags, making sure they are atomic.

This explains a lot.

[mikerj]

But, despite the reams of doc online on the RTOS, even now I can't see a "memory map" anywhere.

evb149 was talking about the map file that is generated by the linker after you compile and link your project.

[peter-h]

Sure; I got that. The .map file doesn't appear to contain any clue to what has been placed at 0x10000000, other than the RTOS 64k memory block which I put there manually.

Actually something strange is going on. I have 16 tasts running perfectly. If I add a 17th one, even the tiniest one, it causes one of the others (an http server actually) to break. I wonder if there is a limit of 16 tasks somewhere? The CPU bombs with a SIGTRAP hardfault handler, and the stack trace, when traced back as far as one can (it shows only 6 unfortunately) shows that the ethernet is trying to transmit a packet of zero length, which presumably is not trapped elsewhere.

And the RTOS does not allocate the whole 64k anyway. There is a good few k unused at the top of the 64k block, evidenced by the untouched 0xAA memory fill. That point moves up and down according to which tasks are enabled and their stack size, as one would expect, but it is safely below 0x1000ffff.

[mikerj]

Is your tiny 17th task initialised after or before the others?  Are you checking the return value from all RTOS API calls that could allocate memory?

[peter-h]

I found the problem

The web server was displaying a list of RTOS tasks. The code for that is some ex-ST demo code. The data was generated, via a hook into "private" FreeRTOS workspace, using a load of sprintf statements, and the buffer needed to hold all of these wasn't "quite" big enough. It was hard coded at 768 bytes.

AFAICT there is no easy way to deal with this, unless one changes the structure of how the data is returned i.e. instead of generating the whole page and returning it to the client, one returns each line of the task listing to the client. I don't know how to do this so just allocated quite a bit extra RAM for it (enough for perhaps 30 tasks) and documented this issue in the bit of code where RTOS tasks are created.

It is otherwise fundamentally nontrivial to know in advance exactly how much data an sprintf could generate, worst-case.

Another happy day's debugging

[bson]

One way to track stuff like this down is to use HW watchpoints.  Set one somewhere at the start of the suspect area and find the code that actually accesses it.

[westfw]

Set one somewhere at the start of the suspect area and find the code that actually accesses it.

A lot of the time, "stuff" gets included that is in fact NEVER accessed in the normal course of events.
For example, the Arduino Due SAM core had an implementation of _exit() that included a printf("exiting...") call (the only one in that "core"), causing 12k worth of stdio code to be sucked into each sketch, whether or not any io device has ever been attached to stdout, and even though _exit() should never get called...  ( https://github.com/arduino/ArduinoCore-sam/issues/47 )

[peter-h]

Well, maybe I missed it, but a simple memory map would have helped a lot.

Apparently the message queues get allocated at the bottom (not quite but they seem to almost start there), then the threads/tasks get allocated as you define them (with the proviso that the order gets a bit funny if a thread starts off other threads in which case, as you might expect, those get allocated last when the whole thing starts up), and eventually you can see how much RAM is left at the end, by looking at what you originally filled the whole block with (I filled with 0xAA, while the RTOS fills each thread's "stack space" with 0xA5 once it starts up).

As per my other thread on the stack sizes, what is left of each thread/task's stack space after running for a bit can be checked properly only by looking at how much of the 0xA5 fill is remaining. A graphical tool which maps out that whole memory block onto a big LCD, with 0xA5 bytes in a different colour, would be dead handy Or even just write out a file to the filesystem if you have one (which I do; might do that) and then somebody in Ukraine, on freelancer.com, can write a little win32 utility which plots that file graphically EDIT: just done the code to generate that 64k file and put the project on Freelancer.

Some funny SP switching takes place because interrupts still use the general stack, so you don't need to leave space on each thread's stack for ISRs - AIUI. Not sure how they do that; must be some ISR time overhead.

[dare]

It is otherwise fundamentally nontrivial to know in advance exactly how much data an sprintf could generate, worst-case.

This is what snprintf is for.  Indeed, any code that uses sprintf to generate a string whose length isn't known at compile time is inherently buggy / dangerous and should be rewritten.

Well, maybe I missed it, but a simple memory map would have helped a lot.

This is possible, but you need to use object initialization methods that avoid dynamic allocation, such as FreeRTOS's xQueueCreateStatic(). This forces you to statically define the memory used by an object, which makes it clearly visible in the .map file.  IMO, this approach is much preferred to a process of manually sampling the heap size (e.g. using your graphical tool), which is always subject to uncertainty.

[peter-h]

You are absolutely right. Doing a malloc and then using sprintf to stuff text strings into it is dumb.

I changed that demo code to use a buffer on the stack (which is ok because one already needs a process to determine RTOS stack space allocation generally) and will change the page generation to use snprintf.

[peter-h]

I got a really great guy in Ukraine (on freelancer.com) to write this graphical presentation, as a win32 executable, of the CCM 64k block allocated to the RTOS. Green are 0xA5 (RTOS task stack free space), yellow is 0xAA (unallocated space at the end), nulls are the squares with a dot in the middle, and the rest are just data but a given byte always looks the same.

1024x1024, representing 64k using 4x4 pixel sprites (256x256 of them). Auto updates from a file which I can write periodically to a USB file system. USD 50.