-
How to change a buffer from main RAM (DATA or BSS) to the heap?
Posted by peter-h on 13 May, 2021 21:34 -
I have this
static uint8_t ucHeap[ configTOTAL_HEAP_SIZE ];
and need to move it to the heap memory. It would be something like
static uint8_t ucHeap[configTOTAL_HEAP_SIZE] = malloc(configTOTAL_HEAP_SIZE);
but obviously that doesn't compile
A stupid Q but no straight answers in many google hits. This is a common example but doesn't compile
uint8_t* ucHeap = (uint8_t*) malloc(configTOTAL_HEAP_SIZE);
with a "error: initializer element is not constant"
I know for a fact that this works, in the same target, and from stepping through I know that dummy acquires the correct address on the heap
void *dummy = malloc(configTOTAL_HEAP_SIZE);
free(dummy);
-
You need to do the call of malloc() inside a function. Global values can only be initialized with constants (or you can use constructors, if you're doing C++)
Edit: change to pointer!
Code: [Select]
int8_t ucHeap*;
int main() {
// initialize heap for uccos right away!
ucHeap = malloc(configTOTAL_HEAP_SIZE);
: -
Surely, the line
int8_t ucHeap[configTOTAL_HEAP_SIZE];
will allocate that in DATA, not on the heap?
Also it doesn't compile, with
warning: type defaults to 'int' in declaration of 'ucHeap' [-Wimplicit-int]
error: conflicting types for 'ucHeap'
warning: initialization of 'int' from 'void *' makes integer from pointer without a cast [-Wint-conversion]
error: initializer element is not constant
-
You wantCode: [Select]
static uint8_t* ucHeap;
int main() { ucHeap = (uint8_t*) malloc(configTOTAL_HEAP_SIZE); -
Wouch. Looks like you need to brush up on your C.
uint8_t* ucHeap = (uint8_t*) malloc(configTOTAL_HEAP_SIZE);
is correct here. (Note that you don't need the (uint8_t*) cast before malloc() in C, malloc() returns a void * which is compatible with any pointer type. In C++, you do need the cast, but then again, using malloc() in C++ is probably just dumb, so trying to make C code using malloc() compilable as C++ doesn't make any sense IMHO.)
But, it is correct in the right context. If you thought of writing the above as a variable declaration at the global level, then it's not correct C. You can't call functions in this context, because global initializers are evaluated at compile time, so you can't use anything that can only be evaluated at run time.
So, if that's what you want to do, you need to declare the pointer at the global level, and then initialize it (with malloc) inside a function, typically one that initializes stuff.
So for instance:Code: [Select]uint8_t * ucHeap;
void MyInit(void)
{
ucHeap = malloc(configTOTAL_HEAP_SIZE);
...
} -
Yes this was a bit dumb. One can't have a "static" buffer on the heap, because the heap is allocated at runtime. As soon as you declare e.g.
uint8_t fred [10000];
that will end up in DATA or BSS and then it is too late to do anything with its address...
Presumably, something declared at runtime with
uint8_t* fred = (uint8_t*) malloc(1000);
can be referenced with e.g.
x = fred[24];
etc.
I solved it with
#define CCMRAM __attribute__((section(".ccmram")))
CCMRAM static uint8_t ucHeap[ configTOTAL_HEAP_SIZE ];
which places it in CCM (STM 32F417). This buffer is used in FreeRTOS which then uses a private "maloc" to allocate blocks to tasks out of it. It would have been a bit silly to have a heap, then use malloc() to allocate this buffer out of that heap, and then have the RTOS using another private "malloc" to allocate blocks out of that. I didn't want to hack the RTOS code around.
Yes I am fairly new to C. Did a few decades of asm
Thank you all.
-
Quote
It would have been a bit silly to have a heap, then use malloc() to allocate this buffer out of that heap, and then have the RTOS using another private "malloc" to allocate blocks out of that.
Not sure why. In fact, to me that looks a quite sensible way to do it (assuming the RTOS doesn't need it before the system has got going enough to enable malloc). -
Easy, use typedef and pointers:Code: [Select]
typedef uint8_t ucHeap_t[configTOTAL_HEAP_SIZE] ;
ucHeap_t *ucHeap;
void main(void){
ucHeap = malloc(sizeof(ucHeap_t)); // Malloc heap, assign pointer to ucHeap
*ucHeap[0]=0; // Write 0 to first heap byte
*ucHeap[configTOTAL_HEAP_SIZE-1]=0xFF; // Write 0xFF to last heap byte
} -
Easy, use typedef and pointers:
Easy, but really, really, wrong.Code: [Select]typedef uint8_t ucHeap_t[configTOTAL_HEAP_SIZE] ;
ucHeap_t *ucHeap;
void main(void){
ucHeap = malloc(sizeof(ucHeap_t)); // Malloc heap, assign pointer to ucHeap
*ucHeap[0]=0; // Write 0 to first heap byte
*ucHeap[configTOTAL_HEAP_SIZE-1]=0xFF; // Write 0xFF to last heap byte
}
Using pointer to arrays in C often leads to surprising results, and this is a canonical example.
Some more detail:- ucHeap is a pointer to an array object, of size 8192 uint8_t (i.e., by defnition, 8192)
- The malloc works fine, as sizoof(ucHeap_t) is 8192
- ucHeap now points to an allocated memory area that can hold a maximum of 8192 bytes
- *ucHeap[0] means "take the first element of the array of ucHeap_t pointed by ucHeap and dereference it.
This works, as the array of ucHeap_t pointed by ucHeap is exactly 1 element long, so ucHeap[0] returns it, the array decays into a pointer to 8192 uint8_t, which is dereferenced, finally (and luckily) yielding the right address. - Now, do the same mental exercise for the following line of code and be amazed.
- remember: [] has precedence over *.
Yes, parenthesis () might help, but it's ugly, unwieldy and cumbersome.
I'll give standard reference chapters and quotes if asked.
-
This is a really weird and twisted way of dynamically allocating memory indeed. Don't do it.
-
I recommend the following pattern:Code: [Select]
TYPE *buffer_data = NULL;Before any operation that starts using the buffer, you call buffer_need(number of elements). It will return the number of elements available in the buffer (which is at least as many as requested), or zero if the dynamic allocation fails; in which case the operation must fail also.
size_t buffer_size = 0;
size_t buffer_need(size_t size)
{
if (size <= buffer_size)
return buffer_size;
free(buffer_data);
buffer_data = malloc(size * sizeof buffer_data[0]);
if (!buffer_data) {
buffer_size = 0;
return 0;
}
buffer_size = size;
return size;
}
void buffer_free(void)
{
free(buffer_data);
buffer_data = NULL;
buffer_size = 0;
}
To conserve memory, when you know you won't need the buffer for a long time, you can call buffer_free(). It is not necessary to call it, ever.
Different operations that cannot be in progress at the same time can use the same buffer. If you have multiple buffers, just use malloc()/calloc()/realloc()/free() for each operation; that's why those functions exist in the first place, and whatever allocator you build on top will usually be less efficient anyway.
The trick to avoiding dynamic memory allocation "costs" is not to avoid calling malloc()/calloc()/realloc()/free(), but to do so when beneficial.
Before anyone asks: free(NULL) is safe to do, and does nothing. There is no reason to check for a NULL pointer before freeing it. -
I think I'm not dereferencing anything. The compiler knows that ucHeap pointers to a ucHeap_t type.
So it knows that *ucHeap[0] is ucHeap address+0, while *ucHeap[n] is ucHeap address+n.
I've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.
But I'm not a professional programmer, I'm willing to hear more!
-
I think I'm not dereferencing anything.
Yes, you are, the variable ucHeap. It just happens to point to an array. Only in this case (*ucHeap)[n] points to the n'th element of that array; not so for non-array types.
Thing is, array indexing has precedence over dereferencing a pointer. That is, *ucHeap[n] is equivalent to *(ucHeap[n]). Which means that *ucHeap[1] does NOT refer to the value of the second element in the array; it refers to the value of the first element in the SECOND ucHeap buffer, at offset sizeof *ucHeap .
Run the following program:Code: [Select]#include <stdlib.h>On my machine, it prints something like
#include <stdio.h>
typedef char block[1024];
int main(void)
{
block *b = malloc(2 * sizeof *b);
if (!b) {
fprintf(stderr, "Not enough memory.\n");
return EXIT_FAILURE;
}
printf("Address of *b[0] is %p.\n", &( *b[0] ));
printf("Address of *b[1] is %p.\n", &( *b[1] ));
printf("Address of (*b)[1] is %p.\n", &( (*b)[1] ));
printf("Address of *(b[1]) is %p.\n", &( *(b[1]) ));
return EXIT_SUCCESS;
}Code: [Select]Address of *b[0] is 0x559eeb17c260.See the difference of 1024 (0x400) between *b[0] and *b[1]? They are NOT CONSECUTIVE IN MEMORY.
Address of *b[1] is 0x559eeb17c660.
Address of (*b)[1] is 0x559eeb17c261.
Address of *(b[1]) is 0x559eeb17c660.
You'd need to use (*b)[0], (*b)[1], and so on, to refer to consecutive elements in the buffer.I've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.
The problem here is that the typedef'd type is an array type. That throws things off.
A very common pattern is to use C99 flexible array members in a structure:Code: [Select]typedef struct {To allocate N data elements, you do
size_t size;
char data[];
} my_char_array;Code: [Select]my_char_array *ma = malloc(N * sizeof ma->data[0] + sizeof (my_char_array));and then you can refer to the elements in the buffer via ma->data[i], for i between 0 and ma->size - 1, inclusive.
if (!ma) {
fprintf(stderr, "Out of memory.\n");
exit(EXIT_FAILURE);
}
ma->size = N; -
I think I'm not dereferencing anything.
You definitely are: an expression such as *ucHeap[n] contains not one, but two dereferences.
The first one is due to the [] postfix operator (C11, 6.5.2.1 Array subscripting) and allow us to rewrite the expression as:Code: [Select]*(*((ucHeap)+(n)))given the definition of [] postfix operator and the fact that it binds stronger than * (see Note)Quote from: DavidAlfaSo it knows that *ucHeap[0] is ucHeap address+0, while *ucHeap[n] is ucHeap address+n.
No, this is patently not true, you are forgetting the two * indirection operators (C11, 6.5.3.2 Address and indirection operators), as described above.
Now let's analyze the rewritten expression for the cases of 0 and configTOTAL_HEAP_SIZE-1 (I set the size to 8192, as an example, my fault not making ti explicit in the answer!).
The variable ucHeap is a pointer to an array of configTOTAL_HEAP_SIZE elements of type uint8_t.
Just to make it absolutely clear: it is not a pointer to uint8_t.
For the case of a 0 offset, the expression is equivalent to:Code: [Select]*(*(ucHeap+0))Obviously, ucHeap + 0 is equal to ucHeap.
The indirection of ucHeap (a pointer to an array) gives us an array of type ucHeap_t.
As arrays decay to a pointer their first element (C11, 6.3.2.1 Lvalues, arrays, and function designators, §3), we obtain a pointer to the first uint_8 in the malloc'ed memory area.
All is good.
Now let's take the case of configTOTAL_HEAP_SIZE-1, for brevity, allow me to use a literal number for configTOTAL_HEAP_SIZE, e.g. 16.Code: [Select]*(*(ucHeap+15))Where will ucHeap + 15 point?
We know that ucHeap is a pointer.
C11, 6.5.6 Additive operators, §7 tell us that ucHeap is treated as the pointer to the first element of an array of length one of type ucHeap_t: remember, we have defined it as a pointer to an array of 16 uint8_t, not as a pointer to an element of the array (uint8_t).
C11, 6.5.6 Additive operators, §8 tells us that the result of the addition will point to the 15th element of an array of ucHeap_t.
Translating it in term of address, 15*16 will be added to ucHeap, as each element is an array of 16 bytes (uint8_t).
The behavior is undefined already - as we have a created a pointer exceeding the size of the array - even before applying the two indirection oprators.Quote from: DavidAlfaI've seen this a lot of times. Malloc a typedef, then use the pointer to access its elements.
Of course. But not in this way: did you check the godbolt link in my answer and what your code was actually doing?
Note: for convenience, I'm using the cppreference site instead of the standard, as the operator precedence rules are not explicitly spelled out in the standard, rather they are derived from the grammar definitions.
Note 2: I've used C11, but there are no relevant changes in these topics since C89.
EtA:Quote from: DavidAlfaI'm willing to hear more!
You now have both the practical answer from Nominal Animal
and the (boring) language lawyer answer from me, choose your poison!
-
Yup, newbrain is definitely correct here, and using the language used in the C standard. Me, I was just trying to show the practical side, as reality wins over theory. Showing exactly why, based on the C standard, shows that the practical observed behaviour is according to the standard, and therefore as expected.I think I'm not dereferencing anything.
You definitely are: an expression such as *ucHeap[n] contains not one, but two dereferences.
Our two answers are in complete agreement, just look at it from different perspectives. (Assume any differences are my wording errors; me often fail English.)
All that said, I wish I could recall or understand the problems some C programmers have with pointers and pointer dereferencing. I honestly just don't perceive those problems at all anymore, and don't remember if I ever did (nor how); the kinds of off-by-one errors I occasionally make with C programs I make in other programming languages as well. For example, in Python array slicing operations. So, the cause of those is my personal failure of logic or math, and not a property of the programming language used. If I was able to perceive or remember how I myself struggled with the more common pointer stuff problems, I'm sure I could be of better help. Sorry.
-
Holy sh**. I will need some time to process this
What about this?Code: [Select]*(ucHeap+0)=0; // Write 0 to first heap byte
*(ucHeap+configTOTAL_HEAP_SIZE-1)=0xFF; // Write 0xFF to last heap byte -
Our two answers are in complete agreement, just look at it from different perspectives. (Assume any differences are my wording errors; me often fail English.)
Of course they are (I never find anything to object with your code): unless someone is using implementation defined or unspecified behaviour (neither is the case here) the standard will tell you exactly what to expect, and for I.D.B. it also describes the alternatives/ranges.Quote from: Nominal Animal, edits by meI honestly just don't perceive those problems [with pointers and pointer dereferencing] at all anymore, and don't remember if I ever did (nor how);
Same here - I had to unlearn bad C picked up from bad books (might Schildt experience nasal demons in eternity) - my cure was reading (and, I hope, understanding) the actual C standard(s), but I used to have more problems with Pascal pointers!
Ah, slicing in Python: love it, it reminds me of when I dabbed in APL. But I always need more than one try to get it right
-
What about this?
DavidAlfa, I think I see the fundamental problem with your understanding.Code: [Select]*(ucHeap+0)=0; // Write 0 to first heap byte
*(ucHeap+configTOTAL_HEAP_SIZE-1)=0xFF; // Write 0xFF to last heap byte
As I said in my first post, eschewing the use of pointer to arrays is in general a good thing, as their behaviour hides some trap for players of all ages.
I'll pick this apart again, hoping I make myself clearer, using the simpler expressions above.
What is ucHeap?
Its definition tell us that ucHeap is a pointer to an object of ucHeap_t type.
What is an object of ucHeap_t type?
The typedef tells us an ucHeap_t object is an array of configTOTAL_HEAP_SIZE elements of type uint8_t.
What's the size of such object?
Easy, you correctly used in the malloc() statement: it's configTOTAL_HEAP_SIZE times the size of uint8_t, which must be one, hence configTOTAL_HEAP_SIZE.
What happens (looking at actual memory addresses) when we add (or subtract) an integer to a pointer?
The integer is multiplied by the size of the object pointed to and added to (subtracted from) the pointer to yield the new address.
In our case, the offset is multiplied by configTOTAL_HEAP_SIZE, so in the second statement we are adding (configTOTAL_HEAP_SIZE -1) × configTOTAL_HEAP_SIZE to the address contained in ucHeap which is definitely a bit too much!
The only offset (or index, [] is just "syntactic sugar" for pointer arithmetic) you can use with ucHeap is, in fact 0, as there is only one element of ucHeap_t type (also 1, but only if the result is never dereferenced).
The main takeaways are:- A pointer to an array is not the same as the pointer to an element of the array:
Though they might correspond to numerically equal addresses in memory, they are object of different types, and follow the rules accordingly. - Array names, in most contexts (exception: sizeof, and & operator), 'decay' to pointers to their first element, but they are not pointer to arrays (well, unless the element themselves are arrays, as in a int a[10][20]; )
Standard references are the same as my previous post, plus, for array decaying into pointers: C11, 6.3.2.1 Lvalues, arrays, and function designators, §3 - A pointer to an array is not the same as the pointer to an element of the array:
-
Quote
I'll pick this apart again, hoping I make myself clearer, using the simpler expressions above.
Whilst you give excellent detail of what's going on, I think the solutions proposed so far kind of muddy the water. Instead of addressing the intent of the original and why/where it falls over, they put forward detailed solution of a different intent.
Perhaps fixing the original in a minimal way would highlight better what the problem is there. With that in mind...Code: [Select]typedef uint8_t ucHeap_t;
ucHeap_t *ucHeap;
void main(void){
ucHeap = malloc(sizeof(ucHeap_t) * configTOTAL_HEAP_SIZE); // Malloc heap, assign pointer to ucHeap
ucHeap[0]=0; // Write 0 to first heap byte
ucHeap[configTOTAL_HEAP_SIZE-1]=0xFF; // Write 0xFF to last heap byte
}
As I saw it, the intent was to malloc a bunch of memory and then access that memory as an array, which is what this does. The main problem with it was just the invalid use of *, presumably through not grasping what they mean (a common issue when learning this stuff). The solutions proposed, whilst obviously correct, have tried to include the * and thus hidden what the original problem was.
-
I agree with dunkemhigh above, but want to take it further.
In dunkemhigh's minimal example, the size of the dynamically allocated array ucHeap is a compile-time constant, configTOTAL_HEAP_SIZE. What do we gain by having it a compile time constant? Not much, really. The array now resides in the heap instead of uninitialized data, but that's it.
I know OP explicitly asked how to do this. I just think the question itself is wrong. Or rather, that the answer to the question solves the actual problem OP is trying to solve by doing this, but that solution is not the sensible/efficient/correct one.
So, let's explore this stuff a bit further, because I don't think we can hope for OP to describe what they are trying to solve by doing this.
First thing we could do to dunkemhigh's minimal example is add the size as a variable, too:Code: [Select]typedef uint8_t ucHeap_t;What did we win here? Nothing yet. However, what if we don't allocate it before we actually need it? Then, we can initialize the array to zero size (or "not available", whatever you wish), and only allocate it when it is needed. Then, we can also release it when no longer needed, if there are long periods when we do other stuff that does not use ucHeap at all:
ucHeap_t *ucHeap;
size_t ucHeap_size;
int main(void) {
ucHeap_size = configTOTAL_HEAP_SIZE;
ucHeap = malloc(ucHeap_size * sizeof ucHeap[0]);
ucHeap[0] = 0; // Set first heap element (byte) to zero
ucHeap[ucHeap_size-1] = 0; // Set last heap element (byte) to 0xFF
return 0;
}Code: [Select]typedef uint8_t ucHeap_t;However, now the compile-time size setting, configTOTAL_HEAP_SIZE makes not that much sense anymore. Instead of setting a compile-time constant, why don't we just pass the size we need when we start needing the heap at all? That's even simpler, you see:
ucHeap_t *ucHeap = NULL;
size_t ucHeap_size = 0;
void ucHeap_init(void) {
if (ucHeap_size != configTOTAL_HEAP_SIZE) {
if (ucHeap_size > 0) {
free(ucHeap);
}
ucHeap_size = configTOTAL_HEAP_SIZE;
ucHeap = malloc(ucHeap_size * sizeof ucHeap[0]);
}
}
void ucHeap_free(void) {
free(ucHeap);
ucHeap = NULL;
ucHeap_size = 0;
}
int main(void) {
ucHeap_init();
ucHeap[0] = 0; // Set first heap element (byte) to zero
ucHeap[ucHeap_size-1] = 0; // Set last heap element (byte) to 0xFF
return 0;Code: [Select]uint8_t *ucHeap = NULL;See how factoring out the allocation/reallocation logic, the array-utilizing code becomes much more logical?
size_t ucHeap_size = 0;
void ucHeap_need(size_t size) {
// If we already have a large enough heap allocated, we're good.
if (ucHeap_size >= size)
return;
// Resize the existing heap to the size needed.
// Note: realloc(NULL, size) is equivalent to malloc(size).
void *old = ucHeap;
ucHeap = realloc(ucHeap, size * sizeof ucHeap[0]);
if (!ucHeap) {
// Reallocation failed, but the old heap (at old) still exists.
// TODO: Fail, abort, restart, whatever here. Cannot continue.
}
ucHeap_size = size;
}
void ucHeap_free(void) {
free(ucHeap);
ucHeap = NULL;
ucHeap_size = 0;
}
int main(void) {
// We do something, that needs a heap of 5000 bytes.
ucHeap_need(5000);
ucHeap[0] = 0;
ucHeap[4999] = 0xFF;
// Something else needs a heap of 7000 bytes.
ucHeap_need(7000);
// Note, ucHeap[0] is still 0 and ucHeap[4999] is still 255.
ucHeap[6999] = 0xFF;
return 0;
We did create a new problem, though: what to do when the allocation/reallocation fails. The reasons for such a failure are twofold: running out of available memory, and having the available memory fragmented, with allocated memory and freed memory scattered about, so that although the sum total of unused memory is greater than what we need, there isn't a large enough consecutive section we could use.
In cases where the previous contents of the array are no longer needed, we can just destroy the old one, then allocate the new one. That way the C library and OS has more chances of finding a suitable free memory fragment:Code: [Select]void ucHeap_init(size_t size, unsigned char zero)
{
if (ucHeap_size >= size) {
memset(ucHeap, zero, size * sizeof ucHeap[0]);
return;
}
free(ucHeap);
ucHeap = malloc(size * sizeof ucHeap[0]);
if (!ucHeap) {
// Out of memory. Abort, exit, reboot etc.
}
memset(ucHeap, zero, size * sizeof ucHeap[0]);
ucHeap_size = size;
}
Anyway, memory fragmentation is the reason why so many developers consider dynamic memory management "bad", especially in embedded environments.
Using malloc() to allocate a buffer for the lifetime of the process (or until the microcontroller is restarted, for os-less environments) does not suffer from memory fragmentation at all, because such buffers are never free()d.
Yet, if we wanted to, we could just use more sensible data structures, instead of just a Big Dumb Buffer!
A commonly used example is a bucket brigade. A single array is split into sub-arrays of fixed size, so that memory fragmentation will not be a problem – the fragments being the same size. For example:Code: [Select]typedef struct ucBucket ucBucket;The size field in the structure is an unsigned integer between 0 and UCBUCKET_DATA_SIZE, inclusive, the latter being a compile-time constant, so that all allocated buckets take the same amount of memory, and memory fragmentation is no longer an issue. It indicates the amount of data in that bucket. Say you have a very large string, and you wish to modify it in the middle. Instead of copying the rest of the array over, it is sufficient to just modify the buckets that contain the to-be-replaced part, even if the replacement is shorter or longer than the original.
struct ucBucket {
struct ucBucket *next;
size_t size;
unsigned char data[UCBUCKET_DATA_SIZE];
};
The downside here is that instead of continuous array functions, we must traverse the linked list also. For example, to find the index of a specific character, or the number of chars in a bucket brigade:Code: [Select]ssize_t ucBB_find_char(ucBucket *bucket, unsigned char c) {In a multithreaded environment, we also need a mutex (or an rwlock, or a similar locking structure) protecting access to each bucket brigade, so we usually have a separate "handle" structure, which often contains a pointer to the final bucket (for very fast append operations), and sometimes even a reference count, in addition to the mutex and initial bucket pointer. All this is just "library" stuff, however, and not visible to the end developer, who just uses the function interfaces provided.
size_t offset = 0;
while (bucket) {
if (bucket->size > 0) {
unsigned char *p = memchr(bucket->data, bucket->size, c);
if (p) {
return offset + (p - bucket->data);
}
offset += bucket->size;
}
bucket = bucket->next;
}
// Not found.
return -1;
}
size_t ucBB_len(ucBucket *bucket)
{
size_t len = 0;
while (bucket) {
len += bucket->size;
bucket = bucket->next;
}
return len;
}
In freestanding C (without the C library), for example in microcontroller environments, it is possible to not have "malloc" at all, just a statically allocated array of "buckets". Then, completely unused buckets can be put into a separate linked list, so that whenever a new bucket is needed, the first one in that list is used; and when a bucket is no longer needed, it is put back onto that list.
In fact, that's very much how C libraries internally implement malloc(), with just two big differences: the buckets are not of fixed size, and the arrays are not statically allocated, but obtained from the operating system via sbrk or mmap calls (in POSIX systems; similarly in other operating systems), and often are not even consecutive in the process address space. (You can think of the malloc() implementation having a super-malloc, maintaining these "allocation pools" it obtains from the OS. Although the details vary between implementations, as a first rough approximation, that gives a reasonable intuition on how things happen "under the hood".)
In systems programming – operating system services and daemons like web servers –, allocation pools are sometimes used explicitly. For example, if each connected client has their own allocation pool, the memory use per client can be trivially monitored and controlled; and when the client disconnects, all the related memory allocations can be discarded at once, with minimal effort.
It is true that automatic garbage collection has advanced so much, that for typical applications, it can perform better than explicit dynamic memory management. There is a lot of peer reviewed literature on how GC is done correctly, and the Boehm–Demers–Weiser garbage collector is a perfect place to start for a C or C++ developer.
Interestingly, Boehm GC provides support for cords, immutable strings that implement something very similar to bucket brigades above. Instead of a linked list, cords are based on a tree, with leaves containing the data (either C strings or the functional description of the contents).
What can we say based on this wall of text as a summary, then?
Dynamic memory management is a tool among other tools. It makes sense, when it makes sense.
Usually the actual problem you're trying to solve has a better solution you can only find by going in deeper; very much like XY problem.
Having a large array with mutable values can be a completely incorrect solution. More complex solutions can be even more efficient, and as shown by Boehm GC in general and its cords in particular, even more robust and reliable than the simple method. Things that appear to be single continuous entities, do not need to be so "under the hood".
In this thread, most posts have been talking about how to use the ucHeap buffer correctly, but the true question should be, what is it used for?
As it is, we're really discussing here how one can use a wood-splitting axe as a hammer, and not what we should be doing: What is that buffer used for? Why are you using a wood-splitting axe as a hammer in the first place; don't you have more appropriate tools you can use?
The latter question is why I seriously believe Data Structures should be discussed in depth immediately when introducing Pointers in C. Abstract data structures like lists, trees, heaps, disjoint sets, and directed graphs make a lot of problems rather easy to solve. They do not always need dynamic memory management, either; for example, (binary) heaps are usually represented as arrays. Heaps are often used for things like priority queues, or when you need more software timers than you have hardware for, even in embedded environments and freestanding C. -
That's quite a long post, but well worth reading through in its entirety
I would like to say, though, that you might use this for dynamically allocating buffers. Perhaps when some comms function starts up you need some buffers, and then when it's closed down you don't. Instead of having them wasting resources you grab them when you need them and lose them when you don't.
-
All this is nice but probably a bit too much in this context. The main point here was about understanding what arrays are in C, which, for some reason, is something that many "beginners" have trouble with. Probably because this a base type that is not consistent with most other base types. In particular, you can't assign an array to an array, or return an array from a function. Another factor is that an array identifier essentially acts as a const pointer, except for the sizeof operator.
-
I would like to say, though, that you might use this for dynamically allocating buffers.
That's exactly what I suggested in my first post in this thread, above. I even used the name "buffer" there, instead of ucHeap.
Typical example I can think of is (re)using a buffer for reading a configuration file off SD card at bootup, and later on for buffering commands/data between the host and the microcontroller. The latter one is logically an asymmetric pair of buffers (usually receive side larger than send side), but they are not needed yet when parsing the configuration file. Instead of trying to juggle them all in one "heap array", it is better to dynamically allocate the exact structures one needs, then free them when no longer needed; and document the dynamic memory use well. When strictly non-overlapping, there is no risk of memory fragmentation either.
Or, a firmware could be able to communicate with a host via any one of UART, SPI, or I2C. (As an example, consider an intelligent display module.) The configuration could be done at run time, either via a selector pin or by dynamic detection. If various buffers are needed, their relative sizes can be chosen at run time to not exceed supportable limits (keeping enough free memory on a microcontroller for stack); with more generous buffers used when the situation allows.All this is nice but probably a bit too much in this context.
True! Not remembering what those hurdles are, severely hinders my ability to stay on track and be useful.
It does not help that the web is full of crappy examples, tutorials, and even libraries, that "kinda work" (at least as long as you have the exact same development environment, versions, and hardware as their writer), but are definitely not good examples. And it is too aggravating to wade through tons of them, to find the good ones... I always feel a pang of guilt when asked, and I cannot really point to any.