Author Topic: Alternatives to Newlib-Nano?  (Read 12486 times)

0 Members and 2 Guests are viewing this topic.

Offline Sal Ammoniac

  • Frequent Contributor
  • **
  • Posts: 950
  • Country: us
    • Embedded Tales Blog
Alternatives to Newlib-Nano?
« on: September 29, 2016, 08:28:26 pm »
I'm doing embedded C programming using Eclipse/GCC for the ARM Cortex-M4 and have been using newlib-nano for a C runtime library up until now. While debugging some code last night I happened to notice that several nano functions internally call malloc(). I don't think dynamic memory allocation in an embedded environment is a good idea in general, especially when it happens behind the scenes. On rare occasions I'll use malloc() myself, but I make that choice consciously.

Is there another C runtime library compatible with ARM GCC that doesn't use hidden dynamic memory allocations?
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 1657
  • Country: dk
Re: Alternatives to Newlib-Nano?
« Reply #1 on: September 29, 2016, 09:55:40 pm »
I'm doing embedded C programming using Eclipse/GCC for the ARM Cortex-M4 and have been using newlib-nano for a C runtime library up until now. While debugging some code last night I happened to notice that several nano functions internally call malloc(). I don't think dynamic memory allocation in an embedded environment is a good idea in general, especially when it happens behind the scenes. On rare occasions I'll use malloc() myself, but I make that choice consciously.

Is there another C runtime library compatible with ARM GCC that doesn't use hidden dynamic memory allocations?

google seems to indicate that newlib-nano uses malloc in printf and you can get around it by not using it or implementing  your own

 

Offline Sal Ammoniac

  • Frequent Contributor
  • **
  • Posts: 950
  • Country: us
    • Embedded Tales Blog
Re: Alternatives to Newlib-Nano?
« Reply #2 on: September 29, 2016, 10:11:04 pm »
It's more than just printf. I haven't discovered the full extent of nano's use of malloc yet, but intend to.

I'm just surprised that a library that bills itself as a "C library (libc) targeting embedded microcontrollers" uses dynamic allocation at all.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #3 on: September 29, 2016, 10:26:53 pm »
I have been using the extremely small C library which comes with MSP430 GCC for ARM controllers for over a decade:
https://sourceforge.net/p/mspgcc/msp430-libc/ci/master/tree/
It is pretty simple to hack the makefile and some source files so it compiles for ARM.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online rstofer

  • Super Contributor
  • ***
  • Posts: 6898
  • Country: us
Re: Alternatives to Newlib-Nano?
« Reply #4 on: September 29, 2016, 10:53:10 pm »
I think string functions also use the heap.

So, I don't use those parts of the library.  Instead, I build up the functions from code given in "The C Programming Language".  I wind up with enough string and conversion routines to meet my needs.

The portion of the standard C library that I wind up needing doesn't seem to use the heap.

 

Offline Sal Ammoniac

  • Frequent Contributor
  • **
  • Posts: 950
  • Country: us
    • Embedded Tales Blog
Re: Alternatives to Newlib-Nano?
« Reply #5 on: September 29, 2016, 11:18:37 pm »
The only thing that I really need that I haven't already written myself is sprintf. I use this to format log entries. I'll have to search for an embedded-friendly version or write one myself.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #6 on: September 29, 2016, 11:29:41 pm »
May I suggest using snprintf? This has a maximum buffer length parameter so it helps against buffer overflows. Either way most C libraries route anything printf-ish into a standard routine (IIRC) vsprintf.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Sal Ammoniac

  • Frequent Contributor
  • **
  • Posts: 950
  • Country: us
    • Embedded Tales Blog
Re: Alternatives to Newlib-Nano?
« Reply #7 on: September 29, 2016, 11:58:36 pm »
May I suggest using snprintf?

Good point. Actually I do use snprintf rather than sprintf.

I've traced through the code for some library versions of sprintf and some of them are almost unbelievably complex and execute literally thousands of instructions to format a simple string.
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 1657
  • Country: dk
Re: Alternatives to Newlib-Nano?
« Reply #8 on: September 30, 2016, 12:22:25 am »
The only thing that I really need that I haven't already written myself is sprintf. I use this to format log entries. I'll have to search for an embedded-friendly version or write one myself.

googling for tiny printf shoudl give you some hints, like http://www.sparetimelabs.com/tinyprintf/tinyprintf.php
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #9 on: September 30, 2016, 01:14:20 am »
May I suggest using snprintf?

Good point. Actually I do use snprintf rather than sprintf.

I've traced through the code for some library versions of sprintf and some of them are almost unbelievably complex and execute literally thousands of instructions to format a simple string.
True. The regular C libraries have become bloated but the interface still dates back to when C got invented and it is very lean and mean. IMHO one of the most common failures is that people throw all the string functions overboard because printf is too large. A much better solution is to use a small C library combined with tiny printf (as langwadt proposed) and keep the good stuff.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 3123
  • Country: us
Re: Alternatives to Newlib-Nano?
« Reply #10 on: September 30, 2016, 09:46:29 am »
avr-libc has looked promising, has a good license, and seems like it wouldn't be too difficult to port to something else.
I was really put off by what newlib does to implement stdio, compared to avr-libc...  (OTOH, a CM4 is more likely to actually have enough of a filesystem to make the newlib scheme have some advantages...)
 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 170
  • Country: fr
Re: Alternatives to Newlib-Nano?
« Reply #11 on: September 30, 2016, 12:37:10 pm »
Most uses of malloc() are for string functions, and it's not true dynamic allocation. Allocation is only valid during function call to avoid using too much stack.
For me it's fine as no string function shoould be in the critical path anyway.
And it's better to reimplement malloc with more efficient algorithm than reimplementing string function which are the number 1 entry point for exploits.
And remember that we are in 201d, string are not longer char*...
 

Offline legacy

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: Alternatives to Newlib-Nano?
« Reply #12 on: September 30, 2016, 01:01:01 pm »
 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 170
  • Country: fr
Re: Alternatives to Newlib-Nano?
« Reply #13 on: September 30, 2016, 01:10:36 pm »
to be more specific: string are no more ascii nowadays, so implementing anything more than a debug_printf requires a lot of work and testing...
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #14 on: September 30, 2016, 01:15:14 pm »
to be more specific: string are no more ascii nowadays, so implementing anything more than a debug_printf requires a lot of work and testing...
UTF-8 is perfectly backwards compatible with '8 bit' strings but any string operation must be limited to the number of characters which fit in the receiving buffer.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Sal Ammoniac

  • Frequent Contributor
  • **
  • Posts: 950
  • Country: us
    • Embedded Tales Blog
Re: Alternatives to Newlib-Nano?
« Reply #15 on: September 30, 2016, 02:34:49 pm »
 
And remember that we are in 201d, string are not longer char*...

They are for me, but then again I'm a provincial American doing this as a hobby and don't need to support other weird languages. ASCII's good enough for me--fuhgettabout that Unicode malarkey! ;)
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #16 on: September 30, 2016, 02:49:46 pm »
Consider yourself lucky.. in one of my projects I had to support several languages with weird signs so I had to write a UTF-8 to LCD display character set converter. The UTF-8 texts also had to go through the editing and compile stage without getting mangled to avoid cumbersome linking in of the texts as binary object. Fortunately Eclipse and GCC managed to do that.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Brutte

  • Frequent Contributor
  • **
  • Posts: 614
Re: Alternatives to Newlib-Nano?
« Reply #17 on: October 05, 2016, 07:05:29 am »
While debugging some code last night I happened to notice that several nano functions internally call malloc().
Yes, it does, for printf-ing for example. But it calls sbrk only once (at first printf) and allocates ~1kB buffer for parsing printf string. When printf does its job, it just passes the buffer pointer further (to UART, semihosting, etc). This 1kB is not free()-d but hogged till exit(). Speeds up processing. Imagine a newlib-nano without malloc(), that does the printf in byte by byte fashion (like avr-libc)  :palm:
 

Offline legacy

  • Super Contributor
  • ***
  • !
  • Posts: 4415
  • Country: ch
Re: Alternatives to Newlib-Nano?
« Reply #18 on: October 05, 2016, 09:43:12 am »
well, the Oracle of Delphi (who promotes pascal & ADA)
had said that things like *printf* MUST be avoided
like a deadly sin which can cause bad things
like when it rains cats and dogs and even outside the Matrix
they all know (including the Architect and his henchmen) that
it's actually raining a ton of code-bugs due to lazy humans

therefore, every attempt to re-implement
those bad things must be banned and punished
and all of those dissidents and rebels
reduced in bondage and obliged to the hammer & stones
to build pyramids with spiral stairs and elliptical temples
in the name of goddess ADA

had said the wise - don't make the Oracle so angry -
but we still staid in those things because we still had H.O.P.E.
(was it Hold On, Pain Ends!, ain't it? ... even under the whip?)


(kidding, it's my humor  :D )
 
The following users thanked this post: nugglix

Offline Kremmen

  • Super Contributor
  • ***
  • Posts: 1283
  • Country: fi
Re: Alternatives to Newlib-Nano?
« Reply #19 on: October 05, 2016, 11:02:49 am »
I didn't read through every single post so maybe this was mentioned already, but i'll say it anyway:
Have you checked the allocators available with FreeRTOS? It comes with several heap managers that behave differently. heap4.c seems to be the most popular but there are others. The descriptions should be available at FreeRTOS site and the code is validated by use in countless projects.
Nothing sings like a kilovolt.
Dr W. Bishop
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 3123
  • Country: us
Re: Alternatives to Newlib-Nano?
« Reply #20 on: October 05, 2016, 09:11:41 pm »
Quote
Imagine a newlib-nano without malloc(), that does the printf in byte by byte fashion (like avr-libc)
An interesting thought.  On an AVR, the likely next-level destination is a UART or other byte-oriented peripheral, so it probably doesn't make much difference.  On more complex systems (say, those involving an OS), the next level is likely to be a filesystem on some block-mode media, and there WOULD be a significant advantage to an intermediate buffer...  (in fact, avr-libc pretty much leaves out the "filesystem" layer, having their stdio functions work directly on top of "devices.")
 

Offline Brutte

  • Frequent Contributor
  • **
  • Posts: 614
Re: Alternatives to Newlib-Nano?
« Reply #21 on: October 06, 2016, 07:21:03 am »
Quote
Imagine a newlib-nano without malloc(), that does the printf in byte by byte fashion (like avr-libc)
An interesting thought.
Yes, but once you hit
Code: [Select]
printf("Chicken counter: %d\n",chickens);the function calls deeper and deeper internals and once some int2str function produces a char, that single byte is sent to uart, then once it is TXed the UART returns, int2str produces second char, etc,etc...
PITA.
The effect would be that you would never have a complete printf string in memory, only single bytes. That "avr printfing" style would also render semihosting impossibly slow.

The alternative for newlib-nano could have been to allocate the buffer statically (without sbrk) at bss:
Code: [Select]
char stdio_buffer[1024];which has the advantage that it can be tied to designated SRAM address (easier debugging and overrun control).
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 1803
  • Country: fi
  • Embedded SW/HW.
Re: Alternatives to Newlib-Nano?
« Reply #22 on: October 06, 2016, 07:43:42 am »
The only thing that I really need that I haven't already written myself is sprintf. I use this to format log entries. I'll have to search for an embedded-friendly version or write one myself.

googling for tiny printf shoudl give you some hints, like http://www.sparetimelabs.com/tinyprintf/tinyprintf.php

+1 for this. I have used this as a starting point / idea when creating my own printf() and sprintf() functions. My major modification was to pass a pointer to a sprintf struct which contains a pointer to a string buffer (char*) and the maximum length given while calling the sprint(). As each character is produced, the function will check whether there is still room in the string buffer. However, if the pointer is NULL, it will pass the characters to the serial port instead. Small overhead, but has proven handy. It can be tailored using various compilation time ifdefs to exclude the features I do not need.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18524
  • Country: nl
    • NCT Developments
Re: Alternatives to Newlib-Nano?
« Reply #23 on: October 06, 2016, 07:52:48 am »
Regarding (re)directing output: I usually do that in putchar (which is called by any string output function) and if speed is an issue in 'puts' as well.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 1803
  • Country: fi
  • Embedded SW/HW.
Re: Alternatives to Newlib-Nano?
« Reply #24 on: October 06, 2016, 08:00:10 am »
Regarding (re)directing output: I usually do that in putchar (which is called by any string output function) and if speed is an issue in 'puts' as well.

That was how I implemented as well, but it requires passing the pointer throughout the functions. However, one can use a global variable for the buffer struct which reduces the overhead a little but the sprintf will not be re-entrant any more. The printf is never a re-entrant by its nature unless one uses a semaphore.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf