Malloc and it's use in MCUs

[ilbs]

Hello everyone,

I have a project which needs to work in real time and can't
afford to hang or undergo  any access violation issues during
its execution. I typically use malloc/free  extensively!!! Then, I
stumbled on the first few paragraphs of the following article:

(see around the middle of the article under
"Memory Fragmentation")

http://www.design-reuse.com/articles/25090/dynamic-memory-allocation-fragmentation-c.html

Now I am worried! If using malloc in MCUs such as pic32 can cause
memory fragmentation, which may cause our programs to hang, then
why should we ever use malloc for embedded systems...
more over, why do compilers for MCUs even offer the possibility
of memory allocation if using malloc can represent such a risk?

Kowing this, really discourages me as it leads me to wonder why did I
bother learning how to malloc arrays, structures, arrays of structures
and arrays of pointers for??

My question is, should we or not use malloc in MCUs ?

Also, if one allocates say 50k in ram, would it be safe
to malloc 3 or four mallocs of 5k each? I don't see how
any problems can arise with this sort of leeway!

thanks

[andyturk]

Avoid malloc() if you can in systems with limited memory.

If you can't avoid it, you'll be better off if all the allocations are the same size. That'll prevent fragmentation in the heap. The next step up is to look at buddy system allocators.

Oh, and that article is incorrect about C++ requiring dynamic memory allocation--there's nothing in C++ that requires you to call new or malloc.

[mikeselectricstuff]

If possible, don't use dynamic memory at all.
If you need to, if possible allocate it it once and never free it.
If you don't know in advance how much memory you need, how do you know you will always have enough?
Otherwise make absolutely sure you understand how the particular malloc/free implementation in your compiler works, and make sure you can deal with any problems it may cause gracefully ( or recognise that at some point you may just need to do a reset to recover).
One scenario is where you need different size/shape buffers, but at different times and never at the same time - this can be better handled by a large static allocation of a union which provides the necessary buffer types within the same memory.

Every application has different requirements so there is no "right" or "good" answer. You just need to understand what's going on at the low level, and figure out an acceptable way of dealing with it.

non-essential use of Malloc/free is a classic error by poeple coming from a PC type environment who don't understand the limitations of small MCU systems.

[Rufus]

Now I am worried! If using malloc in MCUs such as pic32 can cause
memory fragmentation, which may cause our programs to hang, then
why should we ever use malloc for embedded systems...

Microchip won't supply the source code for their run time libraries so you have no idea how it will perform and is reason enough to not use their heap management functions in reliable embedded systems. They grudgingly provided some source code to dump information from their undocumented heap control structures for C30 which carried over to XC16. I can't see the same in the XC32 distribution.

Some (anally retentive) will state that dynamic memory allocation should never be used in embedded systems. Reality is much less black and white. Dynamic management does not cause programs to hang. Allocation and freeing may take varying times and violate real-time requirements. Allocation may fail when heap becomes fragmented which may or may not be a difficult thing to manage.

The likelihood of trouble depends on allocation sizes, patterns, lifetime, and how the heap is managed.

[Bored@Work]

Some (anally retentive)

It is not anally retentive when you avoid a risk. Unfortunately there is now a generation of "programmers" who give a fuck about robustness and safety.

[Rufus]

Some (anally retentive)

It is not anally retentive when you avoid a risk. Unfortunately there is now a generation of "programmers" who give a fuck about robustness and safety.

It is anally retentive to avoid a risk which doesn't exist. See it all the time. Someone did X and screwed up badly therefore no one shall ever do X again regardless of X being the best solution and not screwing up in 90% of cases.

[Mechatrommer]

Now I am worried! If using malloc in MCUs such as pic32 can cause
memory fragmentation, which may cause our programs to hang, then
why should we ever use malloc for embedded systems...

because its easy. thats what higher level language is all about. other aspect is uniformity or portability between mcu's. they are many "resourcefull" mcu out there and are the right spot for malloc.

more over, why do compilers for MCUs even offer the possibility
of memory allocation if using malloc can represent such a risk?

because the compiler's designer "brilliantly" knew that there are some people out there that will handle malloc (they opted to use instead of not use) with cautious and wisdom on limited resourced mcu.

Kowing this, really discourages me as it leads me to wonder why did I
bother learning how to malloc arrays, structures, arrays of structures
and arrays of pointers for??

assuming the compiler is perfect and bugless! if not, you should worry why you want to use higher level language at all in the first place? learn assembly and get acquainted! then you should know the cutting line by then for a particular system. and if you have experienced things like semantic errors, you'll know the "tools" or "existing bugs" is the least of your problem. the problem is how to use them wisely or not use them at all. the link you provided is one step to become a wiser programmer and i'm not saying i'm wiser enough. did i wander off? sorry. maybe too much time on my hand

non-essential use of Malloc/free is a classic error by poeple coming from a PC type environment who don't understand the limitations of small MCU systems.

more accurately begineers, or who are too young to know "limited resource PC type environment" many years back, or who give a fuck on safety and how things work.

ISTR that there is a variant of malloc that will return NULL if it cannot find one continuous block of memory or maybe it depends on the OS used. but i can be mistaken my memory is rusted already.

[marshallh]

Use structs of structs. Encapsulate your entire memory usage within 1 struct at the end.
It's entirely possible and I've done it, it works great when you have a constrained use case and can use the stack for most things instead of malloc.

[IanB]

An embedded program should ideally be fixed in scope and predetermined in its functions (for example, the console display in a car). As such, it should be possible to predict ahead of time what memory is required for what purposes and to pre-allocate by design for each task.

General purpose memory allocators like malloc and free are designed to support general purpose programs on general purpose computers where the resource requirements are variable and not known in advance. A user of a general purpose program has huge freedom to impose different demands on the program at run time in a way that should not happen with embedded firmware.

So if you find yourself wanting to use malloc for memory management, ask yourself, why can't I predict this need when I'm writing the program, and why can't I design in a suitable buffer size at compile time? (If memory is limited you may need to use overlays of different structures in the same space to handle different tasks of course.)

[Rufus]

Use structs of structs. Encapsulate your entire memory usage within 1 struct at the end.
It's entirely possible and I've done it, it works great when you have a constrained use case and can use the stack for most things instead of malloc.

Seems entirely pointless. The reason for using dynamic memory allocation is to allow the same memory to be used for different things at different times.

[marshallh]

You're right. But it all depends on the application. Your app may never have to switch contexts away from the working set of data. Or maybe it has 3 major operating modes, in which case you could switch overlays at runtime.

[nctnico]

Use structs of structs. Encapsulate your entire memory usage within 1 struct at the end.
It's entirely possible and I've done it, it works great when you have a constrained use case and can use the stack for most things instead of malloc.

Seems entirely pointless. The reason for using dynamic memory allocation is to allow the same memory to be used for different things at different times.

That is precisely what the stack does. In some cases you just don't know how much memory is needed at compile time. You have to do a worst case analysis in such cases but allocating at start and never freeing the memory is a perfectly workable solution. BTW even on embedded Linux systems with no swap and several hundred MB of memory fragmentation can be a problem.

If you need dynamic buffers in a microcontroller an good technique is to use a linked list with pieces (blocks) of memory. A chain of blocks represents a buffer. Actually you can extend this system to defragment the chains in case you need to allocate a contiguous block to emulate malloc. Either way the application must have a fall-back scheme in case the system is temporarily out of memory and the application must be designed so all blocks are freed when no longer in use.

[andersm]

Microchip won't supply the source code for their run time libraries so you have no idea how it will perform and is reason enough to not use their heap management functions in reliable embedded systems. They grudgingly provided some source code to dump information from their undocumented heap control structures for C30 which carried over to XC16. I can't see the same in the XC32 distribution.

The same function is present in the XC32 libraries.

[Rufus]

They grudgingly provided some source code to dump information from their undocumented heap control structures for C30 which carried over to XC16. I can't see the same in the XC32 distribution.

The same function is present in the XC32 libraries.

Well if it is the same function as in XC16 fprintfing to stderr it is not exactly convenient efficient or useful. The source is distributed with XC16 so it can be used as an example for creating your own runtime checks and monitoring. I didn't find source in the XC32 distribution.

[westfw]

why do compilers for MCUs even offer the possibility of memory allocation if using malloc can represent such a risk?

Because sometimes (frequently!), the benefits outweigh the risks.  And frequently the risk can be analyzed into near non-existence.  And there are risks to complicated schemes of non-dynamic memory allocation as well.

(that said, I am frequently horrified at the poor "instrumention" of most memory allocation code.  The malloc() I used most included a "show memory" command that would show every block of dynamic memory, who had allocated it (or last freed it), and much other data.  Most MCUs you can't even find out how much memory is available.)  (Of course, there's overhead to this.  Leading to "malloc_lite()" for those instances where you didn't want to put up with that much overhead.  and "chunks" for when there were a bunch of fixed-size things that you needed to allocate from ISRs.  And periodic rewrites over the span of of the 4000x change in "typical" memory configurations over 20 years when various misbehaviors were discovered.  And bugs in the code that used all of the above, of course.)

[IanB]

It is also not obligatory to use the default memory allocator provided with the system, which is written in a general purpose way to suit all foreseeable requirements. If you have more specific requirements you can write your own allocator that can be optimized for your needs.

[ilbs]

Yes, but guys, what if we are careful not to malloc more than
4 blocks at a time? For example, leaving lots of leeway,
4 blocks of 25 bytes each with a heap size of 10K!!!

I don't see how this scenario can cause fragmentation???

even if the 4 blocks were 1K each but never surpassing 4 blocks
at any given time is still very possible because there will always
be enough room to free 1 to 3 blocks and then subsequently
malloc 1 to 3 blocks !!!

I don't see a problem with this... in any case I have tried stuff
like this and the MCU ran for days without any problems
but I never went more than 4 blocks at a time while
always respecting the proportion between heap size to block size
for example (heap size/block size=10K/1k).... which means
ideally the heap can support 10- 1K blocks where as I am only
using 4-1K blocks!!!!

Unless I am missing something.... I think malloc can be used this
way and be very safe. I thinks it's when we malloc huge blocks of memory
per heap size that creates problems.... for example:

Heap size: 10K
Block size: 3K to 5K
# of Blocks used at the same time: 3

... then I believe we would have issues!!!!

PS. I would use it as for each block can *vary* from 1 byte to 10K bytes.


thanks for replies!
 

[Mechatrommer]

I don't see how this scenario can cause fragmentation???

you are at the mercy of the library internal implementation, there can be additional storage needed internally for housekeeping. if you can look at the library's code then do so, if not then.. code once debug twice.

I don't see a problem with this... in any case I have tried stuff
like this and the MCU ran for days without any problems

running MCU for days is not a good idea to ensure the safety of the code. "simulate" every possible inputs and operations, abuse it and make worst case scenario is the right thing to do. sure there will be compromise for this depending on the severity if the program fails.

[zapta]

OP, you are very right to worry, I also would worry. The fact that malloc exists does not mean that it is good for your application.

Your best bet is to allocate your resources in advance (you can static structs with arrays, etc, more memory efficient than malloc BTW). If you need to dynamically allocate items (e.g. new items for lists), implement your own 'allocation' over the item pools you created during initialization, e.g. have a list of free items and move between lists. If your data is of variable type/size, you can have multiple pools or make them unions as Mike suggested.

Beware also of library functions/classes that may use malloc/new post program initialization, they are just as risky.

The C heap cannot compact itself so it is non deterministic in general (unless if you prove somehow that your specific allocation/release pattern will never fail).

[free_electron]

MALLOC is an instruction that should ONLY be used on systems that dynamically load a runtime.

A microcontroller system does not fall in this category. A microcontroller system runs from a rom. This predicates that the memory layout is KNOWN at compile time.

[nctnico]

Like I typed before: the memory layout is not always known at compile time. It can depend on the system's configuration. Malloc once and never free is a perfectly valid technique.

A much better way would be to use a C++ compiler. C++ allows to allocate arrays which are sized at runtime and cleanup up automatically when they go out of scope. Still you'd need to do a carefull examination of the worst case memory requirements.

[andyturk]

Malloc once and never free is a perfectly valid technique.

Yup.

C++ allows to allocate arrays which are sized at runtime and cleanup up automatically when they go out of scope. Still you'd need to do a carefull examination of the worst case memory requirements.

In a single-threaded system, your heap and stack will share a boundary so it's not such a big deal. But if you're using multiple threads, you'll need to know how much space you need for each stack and that gets complicated quickly.

[ejeffrey]

Another option for many c implementation is alloca.  It allocates a dynamic length buffer on the stack, and automatically frees it when it goes out of scope.  You still have to be careful: you can easily run out of stack, but you won't have memory leaks, and you can have some wiggle room in knowing how big your structures need to be at compile time.

Also, be warned that avoiding dynamic memory management can also destroy your program.  If you have heard of 'buffer overflows', they used to be responsible for the majority of security breaches on the internet.  In most cases, those were due to people using fixed length buffers because it was easier than dynamic allocation, then one day a bit of data came along that was bigger than they could handle.  Crash, and possible security breach ensues.  Dynamic languages and use of high level libraries to handle dynamic memory management in languages like C and C++ have mostly eliminated these problems in the PC world and made way for new and more complicated security breaches, but lots of embedded software is still designed that way.

So basically, you are damned if you do, damned if you don't.  Software development is hard.  Programs are complex, and anticipating every possible circumstance and every possible state is generally impossible.  You have to do the best you can to anticipate everything, then code defensively around the assumption that you didn't.

[westfw]

Use of a data stack should be avoided in any microcontroller architecture that is incapable of detecting stack overflows and underflows of that stack.  Preferably, avoid stacks unless you can allocate substantial areas of inaccessible memory other either side of the stack, so that buffer overflows of local variables can be trapped.  In addition to buffer overflows, inadequately tracked call depth can cause data stack areas to collide with static data.  All data should be pre-defined; allowing "local" data to be allocated on a stack is no better than allowing malloc().   If your processor lacks a dedicated hardware stack for use as a subroutine call mechanism, it should use some other mechanism for subroutine calls.  Some HLLs have "goto" and "computed goto" statements that can be used (with some care) for subroutine calls; avoid HLLs that insist on using a stack for such purposes.

Note that modern CPU architectures (ARM, MIPS) DO support subroutine linkage schemes other than stacks.  If we're going to have reliable embedded firmware, we have to mate the architectures with better HLLs!

 :-)

[nctnico]

You can check the stack with every microcontroller. If you sample the stack pointer in a timer interrupt you can see if it has advanced too far and take action. Its a bit like a random time sampler but after a while it will get a worst case value.

[Rufus]

Why are people talking about stack?

The lifetime of stack and heap allocations is completely different.

[AlfBaz]

Another problem I've seen with a lot of malloc implementations is that they don't care what size you allocated to your heap and will keep growing upwards toward the stack. Most of the time defined heap size is solely for the linker to throw an error if heap size and stack size (which grows downward) overlap during compilation. Also A lot of stack implementations don't give a shit about their size restrictions either

[denizcan]

don't be afraid of malloc. just know how it behaves. sometimes it makes life simpler. it is there because it makes memory shareable..

suppose one module requires some memory for a calculation/task. but it releases used blocks afterwards. and another module also requires some memory for some other calculation/task, but it also does not need that much memory after the task is completed. and also suppose that both of those modules do not run simultanuously..

how would you manage that problem? one solution is static allocation of some memory and suplying it to the modules.. ok, you have to code some for that. the other is using malloc and free.. if you free all you allocated, then there is no fragmentation problem.. this is why heap is there on embedded systems. dynamic functions that require different sizes of memory..

one example is printf.. you gave some string to format the output. heap makes printf code more efficient, more elegant.. it creates some buffer to hold the formatted string, and passes it to put char or similar function.

if you are working on kb of buffer, again and again, I suggest you preallocating that much buffer in a linked list. it is a lot fater, and predictable..

I prefer C++, in C it is alot uglier.. such that:

Code: [Select]

typedef struct _MyStruct
{
_MyStruct* next;
int* myData[2000];
} MyStruct;

MyStruct* emptyStructs;

MyStruct* takeOne()
{
MyStruct* result = emptyStructs;
if (emptyStructs)
emptyStructs = emptyStructs->next;
return result;
}

void releaseOne(MyStruct* s)
{
s->next = emptyStructs->next;
emptyStructs = s;
}

MyStruct structs[5];

int main()
{
int i;
emptyStructs = NULL;
for (i = (sizeof(structs) / sizeof(MyStruct)) - 1; i; i--)
releaseOne(&structs[i]);

MyStruct* s = takeOne();
....
releaseOne(s);
}


In C++ you may define LinkedList template class which handles this.. I use it frequently. For instance in GUI the system should calculate clip map (the visible rectangles of widgets). It uses pre allocated list of records.. With template:

Code: [Select]

LinkedList<ClipRectangle> freeRectangles;
ClipRectangle freeRectanglesBuffer[5];
freeRectangles.append(freeRectanglesBuffer, sizeof(freeRectanglesBuffer)); // add to last

LinkedList<ClipRectangle> visibleMap;
ClipRectangle r = freeRectangles.removeFirst();
// calculate r
visibleMap.prepend(r); // add to front
etc etc
...

//if you want to free a data:
freeRectangles.prepend(r); // adds to the front of the list
// or if you want to free all visibleMap list
freeRectangles.prepend(visibleMap);

isn't it a lot neater?

[andyturk]

isn't it a lot neater?

Good onya for posting code. 

The $64K question is what happens when LinkedList<T>::removeFirst() is called and the list is empty. That's essentially the same error condition as when malloc() fails. The code you posted doesn't appear to check for this, and I'd guess the code would crash if it ever needed a 6th rectangle.

I've used explicit free lists for bigger objects (e.g., buffers in a networking stack), but the overhead of actually checking and dealing with each different kind of out-of-memory error means you probably wouldn't want to take that route for smaller objects.

[denizcan]

removeFirst returns NULL.. there are different approach on memory management handling..

1) allocate extra memory.
2) if you are using ARM, use memory protection of it. it will give you hardware fault in first write attemt.
3) check if return is NULL, or check if freeRectangles.hasItems(), or check freeRectangles.getCount() is grater than zero.

I don't prefer last method because it is unnecessary code until the list is depleted.. In embedded, if you depleted memory, that means one of those: you allocated less, you have a leak, you didn't designed well.. You allocate item to write on it. At that point ARM will take you to hard fault handler, there you can observe, or better log the error with stack to flash and reset, or wait and watch dog resets the CPU.. That way I found memory handling errors in the remote field.. I suppose you tagged in your version control what you sent to the field..

Memory protection makes life a lot easier.. What would you do if your list is depleted? How can you solve a leakage by cheking..

[denizcan]

By the way, just to add: for performance wise, think about a windowing system that has to calculate clipping regions when layout changed. That means thousands of comparison, rearrangement, calculation, list manuplation etc etc. For responsive GUI it must be as quick as possible.. One comparison counts. Keil's crappy compiler generates load, compare, branch if not instructions.. comparing with NULL, it always executes a jump. in that time I would decide complete some of intersection code.

think about this: to optimize I sometimes convert while loops to do while loop and copy the comparison

Code: [Select]

while (condition)
{
}

generates a jump, comparison, jump.. two extra jumps.. n + 2 jumps.

Code: [Select]

if (condition)
  do
  {

  } while(condition);

does not jump if condition is true. jumps if false. it also jumps if while condition is true, does not jump at last loop.. n - 1 jumps for n >= 1, 1 jump for n == 0.. jumps are expensive.. I am surprised to see in 2013 compilers don't generate optimized loops, even if you set "optimize for time".. when this is meaningful? usually when dealing with graphics intensive stuff. suppose you are putting lots of pixels.. even one instruction counts.. that way, if you are using external memory controller to push pixels to display, you may calculate pixels much faster than display can take..

[IanB]

By the way, just to add: for performance wise, think about a windowing system that has to calculate clipping regions when layout changed.

IMHO, a windowing UI system of this kind is stretching the concept of "embedded firmware" beyond reason (assuming we are talking graphics and not simple text boxes). For such systems you need an OS. It may be embedded, but it is not firmware on a micro, it is a full blown small scale computer.

[denizcan]

IMHO, a windowing UI system of this kind is stretching the concept of "embedded firmware" beyond reason (assuming we are talking graphics and not simple text boxes). For such systems you need an OS. It may be embedded, but it is not firmware on a micro, it is a full blown small scale computer.

Not all products needs that much complexity. Nowadays a 10$ chip has 1M memory, nearly 200K RAM, @ ~170MHz, with floating point etc. It also has memory controller which makes it very easy to transfer the pixels from RAM. Even some chips have tools to help drawing.. This is more than enough power to handle complicated graphics. Also this is more powerful than 80286 we were using when the GUI era started..

If you want double buffering, you add some more external RAM for a few dollars, then you have very sleek, ultra responsive, user interface.. It is also boots up instantly as it does not load operating system.. It all resides in the design.. Operating system is not necessary, however good RTOS helps responsivity.

[grumpydoc]

isn't it a lot neater?

Your C++ snippet misses the template implementation details which you include in you C example and neither have any "out of structs" handling so it's a bit hard to tell. However if we ignore the actual implementation and initialisation then it boils down to

Code: [Select]

MyStruct* s = takeOne();
....
releaseOne(s);

vs

        ClipRectangle r = freeRectangles.removeFirst();
        ...
        freeRectangles.prepend(r);

So it's probably pretty close in this case.

I have to say though that in many ways I'm not a big fan of C++ (even though I've written a lot of C and even more C++) because it's all too easy to a) write unmaintainable code and b) get screwed by language features.

Case in point - the code example you could easily not be doing what you think.

OK, it was a quick "illustrate the point" snippet and I'm not going to hold you to it actually working, or even compiling but picking it apart slightly the line

Code: [Select]

ClipRectangle freeRectanglesBuffer[5];
Suggests (only suggests mind you since we don't see the implementation) that the type of ClipRectangle is the class, not a pointer.

So

Code: [Select]

ClipRectangle r = freeRectangles.removeFirst();

could be a bit of a problem - it instantiates a new automatic ClipRectangle (calling its default constructor), then assigns it (calling op =) from the result of removeFirst() - rather a lot of copying will be going off (not great for efficiency) and more importantly you don't now have the one you thought you did - just an automatic variable holding a copy of the data.

Now

Code: [Select]

freeRectangles.prepend(r);

has the potential to bite you badly. For this to compile there would have to be a version of prepend() which took a ref - that's possibly not unreasonable depending on how you use the lists but you might just have stuck an automatic variable into a static list. Depending on the lifetime of the variables you might get away with this to some extent but you might not and the bugs produced are often difficult to track down.

Now the chances are that your template is pointer based and you meant to say

Code: [Select]

ClipRectangle *r = freeRectangles.removeFirst();
or even

Code: [Select]

ClipRectangle &r = freeRectangles.removeFirst();

There is a good chance that the compiler would have spotted the mismatch and thrown an error, especially in a fairly simple project which doesn't have loads of overloaded methods and multiple constructors but I've seen real code die on exactly the above mistake in a project where the compiler was able to "helpfully" figure out a way to make the by value assignment work where a by pointer assignment was intended.


Don't get me wrong, I love C++ for writing code. In fact I'd probably love it even more today than when I last seriously (i.e got paid for doing so) wrote a significant amount of C++ and several interesting features have been added to the language since then.

What I loath (and that's with a vengeance) C++ for is maintainability in all but the simplest of projects, especially when all the "nice" language features get used. It can be totally impossible to know what even the most innocuous of lines of code are actually going to do.

[denizcan]

thank you spending time to examine and write detailed answer..

ClipRectangle something like that

Code: [Select]

class ClipRectangle
{
friend class LinkedList<ClipRectangle>;
private:
ClipRectangle* next;
...
};


Code: [Select]

ClipRectangle r = freeRectangles.removeFirst();
 ...
 freeRectangles.prepend(r);

I mistakenly forgot *.  It is actually ClipRectangle* r. Of course it does not compile.

Code: [Select]

        ClipRectangle* r = freeRectangles.removeFirst();
        ...
        freeRectangles.prepend(r);


Code: [Select]

freeRectangles.append(freeRectanglesBuffer, sizeof(freeRectanglesBuffer))

append, and prepend adds the array of ClipRectangle's to linked list. They actually bind next fields together and updates it's first,last pointers according to.. I added this to template, because in embedded I am using arrays to allocate memory statically.

The similar pattern is used in Task, Mailbox, Dispatcher classes etc:

Code: [Select]

UWord taskStack[128]; // I prefer UWord type array but one may also use Byte
Task task(taskStack, sizeof(taskStack));

etc. freeRectangles.removeFirst() does not allocate anything. it just returns what first pointer points, and steps first = first->next..  I don't think C++ creates problems more than C.. There is two different sides of development for me, library development, and application development. My time is mostly spend on library very carefully, which makes it possible to write application very fast in a very short time. The customer usually gets a few pages of code. So, I can say C++ saves my time..

[amyk]

I avoid dynamic allocation even in non-embedded systems, since unless it's absolutely necessary all it does is add complexity. Often I spend a little more time thinking how the same thing can be done without it, and the resulting algorithm turns out simpler and more efficient.

The prevalance of stuff like "malloc(1000)" followed by a few lines and then a free() in publicly available code I've seen suggests that the majority of programmers these days have little idea what they're doing, but that's a rant for another day...

[denizcan]

I avoid dynamic allocation even in non-embedded systems, since unless it's absolutely necessary all it does is add complexity. Often I spend a little more time thinking how the same thing can be done without it, and the resulting algorithm turns out simpler and more efficient.

The prevalance of stuff like "malloc(1000)" followed by a few lines and then a free() in publicly available code I've seen suggests that the majority of programmers these days have little idea what they're doing, but that's a rant for another day...

For non embedded systems, how do you manage unknown amount of data? For instance you are going to load a bitmap, a wave file, a list, or make a database query..

[grumpydoc]

For non embedded systems, how do you manage unknown amount of data? For instance you are going to load a bitmap, a wave file, a list, or make a database query..

If you write "ptr = malloc(1000)" then you know the size you want and could probably do better with an automatic varable of the correct size/type.

If you write ptr = malloc(dynamically_determined_size) then fair enough.

[mazurov]

Another way to allocate dynamically is to use memory pools. One good example of this approach is QP framework, see www.state-machine.com.

[Rufus]

For non embedded systems, how do you manage unknown amount of data? For instance you are going to load a bitmap, a wave file, a list, or make a database query..

If you write "ptr = malloc(1000)" then you know the size you want and could probably do better with an automatic varable of the correct size/type.

If you write ptr = malloc(dynamically_determined_size) then fair enough.

alloca will give you locals of 'dynamically_determined_size' but is almost pointless. You either have enough stack for the largest 'dynamically_determined_size' or you don't. Only when you have multiple allocations and the knowledge that the combined size is less than the sum of individual maximums does only allocating what you need have any benefit.

The requirement to not use stack allocation is determined by the required lifetime of the allocation not size.

[grumpydoc]

alloca will give you locals of 'dynamically_determined_size' but is almost pointless

It's not terribly portable either I would not recommend its use.

This is one place where C++ scores heavily over C because automatic variables have their destructors called when they go out of scope so you don't have to worry about failing to free acquired resources (such as heap memory) or have convoluted paths to cope with cleaning up in C.

That said in C this is one place that use of goto can actually improve readability - I have no particular problem with

Code: [Select]

int somefunc()
{
    int retval = 0;
    sometype *ptr1 = 0;
    anothertype *ptr2 = 0;
    somestruct *ptr3 = 0;

   ...
   ptr1 = malloc(sizeof(sometype));
   ...
   ptr2 = malloc(sizeof(anothertype)*this_is_how_many_I_need);
   ...
   ...
   if (really_horrendous_error) {
        retval = ERR_SOMETHING_WENT_WRONG;
        goto cleanup_and_return;
   }
  ...
   ptr3 = malloc(sizeof(somestruct));
   ...
   ...

    cleanup_and_return:

    if (ptr1) free ptr1;
    if (ptr2) free ptr2;
    if (ptr3) free ptr3;
    return retval;
}

but C++ makes it easy to write much neater code for safe resource de-acquisition

[edavid]

alloca will give you locals of 'dynamically_determined_size' but is almost pointless

It's not terribly portable either I would not recommend its use.

Name one significant platform that doesn't support it.

[grumpydoc]

Name one significant platform that doesn't support it.

Alloca portability is more about the detailed behaviour than simple availability. It isn't part of any standard so the behaviour you get from it varies. In Microsoft C you get a structured exception if you try to extend the stack beyond its limit, in linux the man page tells me you probably don't get any exception but then are at risk of SEGV's having broken the stack. Other platforms (eg NETBSD) might return 0 for a failure.

C99 gives you dynamically sized arrays which should be chosen if available as they are standard and have defined behaviour.

[zapta]

One disadvantage of alloca is that space is not shared among threads. If you have a pool of N threads that may call a function with alloca(some_size) than you need to reserve the size on each of the threads' stacks.

It has other issues as well so I try (and manage) to avoid it.

Here is an interesting gotcah with alloca

http://stackoverflow.com/a/3410689/1139880

[amyk]

I avoid dynamic allocation even in non-embedded systems, since unless it's absolutely necessary all it does is add complexity. Often I spend a little more time thinking how the same thing can be done without it, and the resulting algorithm turns out simpler and more efficient.

The prevalance of stuff like "malloc(1000)" followed by a few lines and then a free() in publicly available code I've seen suggests that the majority of programmers these days have little idea what they're doing, but that's a rant for another day...

For non embedded systems, how do you manage unknown amount of data? For instance you are going to load a bitmap, a wave file, a list, or make a database query..

Unless it's absolutely necessary to have all the data in memory for some reason, just use an algorithm that doesn't require that. If you're loading a bitmap for display, and the display has its own memory, there's absolutely no reason you need to read the whole thing in at once - read in blocks of 512 bytes or some other convenient size and write them to the display. Even with non-embedded systems e.g. working with huge images or video files this is a very useful technique.

[Mechatrommer]

a purely technical thread can surely bloat (if not trolled)  and why i'm even here?

[zapta]

I avoid dynamic allocation even in non-embedded systems, since unless it's absolutely necessary all it does is add complexity. Often I spend a little more time thinking how the same thing can be done without it, and the resulting algorithm turns out simpler and more efficient.

Check the 'premature optimization' comments here http://en.wikipedia.org/wiki/Program_optimization#When_to_optimize .  Not all the lines code are in the inner loops of CPU intensive algorithms.

The prevalance of stuff like "malloc(1000)" followed by a few lines and then a free() in publicly available code I've seen suggests that the majority of programmers these days have little idea what they're doing, but that's a rant for another day...

In most cases it's just fine.  Alloca triggers red flags about stack size. Static alloc is not reentrant and suggests persistence across invocations. I don't think that majority of open source programmers these days have little idea what they're doing.