Alternatives to Newlib-Nano?

[SiliconWizard]

printf(), at least in its original implementations, is basically a character-by-character function calling putc() for each character. It doesn't need any dynamic allocation per se except a few local variables AFAIK.
I haven't looked at a "modern" implementation of it in ages, so some libs may use dynamic allocation to speed things up, but I have no clue. It's not really necessary. (Reminds me of a discussion we had on dynamic allocation though. Depends on what we call that. Using local variables and function parameters is still some kind of dynamic allocation. So I guess we should add "on the heap".)

As for using it in embedded contexts - I think I've never used printf() directly in any embedded software in ages. Admittedly I do use sprintf() or better, snprintf() when available and mostly for non-timing critical tasks such as UI needs. I don't see the point of using printf() and having to define a custom putc(), as using snprintf() and the likes and then sending the resulting buffer to whatever peripheral will be much more efficient since you can take advantage of the specifics of the peripheral in terms of buffering (FIFO, DMA...), something you can't with putc() AFAIK. Some might argue that printf() allows a non-limited buffer size by nature, and it's true, but I actually don't like that, especially in embedded settings. Far too slippery for my taste. I prefer bounding every transaction that I issue.

That said, in terms of performance or code size, yes sprintf() does basically the same thing as printf() except it doesn't need to call a callback function like putc(). I wouldn't use it when I need performance. printf() and its siblings is very inefficient (although admittedly very flexible and useful) compared to implementing direct formatting functions as needed.  Even on a fast PC. Just test a sprintf() call with a moderately complex format in a loop and time it. Compare to the same thing implemented with direct formatting. Of course if neither space nor timing is critical, have at it. But then avoid sprintf() if you can and use snprintf() and the likes instead. Avoids the possibility of nasty buffer overflows.

Other basic functions from the standard lib are much more efficient and take up little space, such as memcpy(), strcpy(), strcat(), memset(), memcmp()... With optimizing compilers, they are even often compiled as direct and very efficient inlined code. So there's no reason to refrain from using those.

[richardman]

Obviously printf and fprintf (vprintf and vfprintf) use malloc but sprintf and snprintf (vsprintf and vsnprintf) don't. strdup does as per specified by its contract as a chained call of malloc and memmove. Some implementation of sscanf (vsscanf) uses malloc for internal buffers, and if so through that fscanf and scanf (vfscanf and vscanf) will also use malloc.

Yes of course strdup by definition uses malloc. The OP though said STRING functions, and we can all agree that printf, sprintf, etc. are stdio.h functions 

In other words, I am not convinced of the OP's claims. I think other than printf and friends, and strdup, nanolibs or any C library implementation, for that matter, should or WOULD use malloc.

*as a compiler writer, I have written my share of C library functions*

[technix]

Obviously printf and fprintf (vprintf and vfprintf) use malloc but sprintf and snprintf (vsprintf and vsnprintf) don't. strdup does as per specified by its contract as a chained call of malloc and memmove. Some implementation of sscanf (vsscanf) uses malloc for internal buffers, and if so through that fscanf and scanf (vfscanf and vscanf) will also use malloc.

Yes of course strdup by definition uses malloc. The OP though said STRING functions, and we can all agree that printf, sprintf, etc. are stdio.h functions 

In other words, I am not convinced of the OP's claims. I think other than printf and friends, and strdup, nanolibs or any C library implementation, for that matter, should or WOULD use malloc.

*as a compiler writer, I have written my share of C library functions*

From a maintainer's aspect, using malloc in fprintf (and in turn printf) makes sense, as it means that there would be only one instance of printf code in vsnprintf and all other whatever-printf functions directly or indirectly calls it. A malloc-less version would have to include two copies of almost the same code in vsnprintf and vfprintf respectively, or use some function pointer magic and slash vsnprintf efficiency almost in half.

[SiliconWizard]

From a maintainer's aspect, using malloc in fprintf (and in turn printf) makes sense, as it means that there would be only one instance of printf code in vsnprintf and all other whatever-printf functions directly or indirectly calls it. A malloc-less version would have to include two copies of almost the same code in vsnprintf and vfprintf respectively, or use some function pointer magic and slash vsnprintf efficiency almost in half.

I beg to differ. They just have to call a 'putc' function instead of writing to the given buffer's current pointer and incrementing it. Just one test.

[technix]

From a maintainer's aspect, using malloc in fprintf (and in turn printf) makes sense, as it means that there would be only one instance of printf code in vsnprintf and all other whatever-printf functions directly or indirectly calls it. A malloc-less version would have to include two copies of almost the same code in vsnprintf and vfprintf respectively, or use some function pointer magic and slash vsnprintf efficiency almost in half.

I beg to differ. They just have to call a 'putc' function instead of writing to the given buffer's current pointer and incrementing it. Just one test.

Technically this can be done: (Please be patient while reading the code.)

Code: [Select]

ssize_t _real_vnprintf(ssize_t limit, void (*action)(void *context, char ch), void *context, const char *format, va_list args)
{
  while (...)
  {
    char ch;
    // Actual printf code
    action(context, ch);
  }
}

struct _vsnprintf_context
{
  char *buffer;
  off_t *location;
}

void _vsnprintf_action(void *context, char ch)
{
  struct _vsnprintf_context *ctx = context;
  ctx->buffer[*ctx->location] = ch;
  *ctx->location++;
}

ssize_t vsnprintf(char *buffer, size_t limit, const char *format, va_list args)
{
  struct _vsnprintf_context ctx = {buffer, 0};
  ssize_t rv = _real_vnprintf(limit, _vsnprintf_action, &ctx, format, args);
  ctx.buffer[ctx.location] = 0;
  return rv;
}

ssize_t vfprintf(FILE *file, const char *format, va_list args)
{
  return _real_vnprintf(-1, fputc, file, format, args);
}The overhead of calling _vsnprintf_action for each character will add up - not ideal for slow CPU. Or if _real_vnprintf is a macro the resulting code will have two copies of printf code, one for vsnprintf and one for vfprintf - not ideal when Flash memory is constrained.

The implementation of vfprintf through malloc and vsnprintf allows those two scenarios to be averted: there is only one copy of printf code in vsnprintf, and that is the fastest possible version. newlib seemed to me lacked dynamic stack allocation which is usually a better option for this though.

[SiliconWizard]

I can definitely think of a couple ways of implementing just one core function for all the printf() variants and use it to implement them without duplicating code. Definitely doable without code duplication.

[technix]

I can definitely think of a couple ways of implementing just one core function for all the printf() variants and use it to implement them without duplicating code. Definitely doable without code duplication.

I had put up an example above, but that code uses function pointers and reduces efficiency of vsnprintf significantly due to all the function call overheads. The best solution I know of would be using a dynamic stack allocation (similar to malloc but the memory is allocated on stack and freed when exiting the current stack frame.) which also builds vfprintf using a fast vsnprintf like that malloc example above but does not rely on malloc or sbrk.

[richardman]

Speaking as a compiler vendor who has multi-thousands of users since 1994 for MCU from HC11, HC08, CPU12, ARM V7, Propeller, MSP430, Cortex... we have always used a single core printf-format function and use function pointers to either write using putc, or stuff the output to a buffer, depending on whether it is printf or sprintf (and the variants).

Indeed, for the smaller MCUs, it's a worse sin to have multiple similar code functions than printf being slow as customers can easily understand that printf can be and will be slow, due to its features.

[technix]

Speaking as a compiler vendor who has multi-thousands of users since 1994 for MCU from HC11, HC08, CPU12, ARM V7, Propeller, MSP430, Cortex... we have always used a single core printf-format function and use function pointers to either write using putc, or stuff the output to a buffer, depending on whether it is printf or sprintf (and the variants).

Indeed, for the smaller MCUs, it's a worse sin to have multiple similar code functions than printf being slow as customers can easily understand that printf can be and will be slow, due to its features.

I had the misfortune of having to work with some folks that really loves to use printf, and they need a fast implementation of printf with a limited memory space.

[Karel]

C as a language is broken anyway, why bother. Yes, you can pretend that C will be a safer with some constructions and limited subset, but nevertheless the C as a language is unsafe.

soldering irons are unsafe, there is only  limited number of ways you can use one without breaking things or burning you fingers

C language is not the problem. C is a compromise between overhead/performance and
abstraction level. C does require an experienced programmer who knows what he is doing.

Or is a surgeon with a scalpel also a problem?

[legacy]

C language is not the problem

If the C language wasn't the problem, we wouldn't have the need to schedule up to the 45% of the activities to make it MISRA compliant and then to make it DO178B-level{A-C} compliant. I can assure you nobody is happy to spend his paid time for these activities since they are very boring and frustrating.

Can't those developers write their code already compliant? No, with C, usually they can't, but in avionics, we don't need to schedule so many % of activities with everything is written in Ada.

Unfortunately, we can't write the firmware of our AFDX, ARINC, and stuff in Ada, which is only used for everything is on the top of the BSP, iBSP, and OS.

I don't want to be put in the same basket of those who are arrogant, but personally, I think the C language carries the design and philosophy of the 80s, whose attempts to fix, C89 and C99, have helped, but not resolved the problem.

[Karel]

C language is not the problem

If the C language wasn't the problem, we....

I understand your worries when writing software for safety-critical purposes.
For all other purposes where you don't want slow and bloated and energy consuming software,
C is a good choice for motivated programmers who understand the internals of computers and electronics.

Anyway, whether you like it or not, C is here to stay and it's not going anywhere in the foreseeable future.
In fact, it's one of the most used programming languages. (only Java is more used)

[legacy]

C is here to stay

I am not sure about this. There are other languages in development, and for example, The Rust Programming Language is very interesting and promising 

[JPortici]

C is here to stay

I am not sure about this. There are other languages in development, and for example, The Rust Programming Language is very interesting and promising 

But until the MCU manufacturer or somebody else come up with a certified rust (or whatever language) compiler for said MCU, I (as in developer) am going to what's available, which is usually a C compiler, which is why C is going to stay for a long time..

[SiliconWizard]

Speaking as a compiler vendor who has multi-thousands of users since 1994 for MCU from HC11, HC08, CPU12, ARM V7, Propeller, MSP430, Cortex... we have always used a single core printf-format function and use function pointers to either write using putc, or stuff the output to a buffer, depending on whether it is printf or sprintf (and the variants).

Yes, that's pretty much what I was meaning. I don't think there is another sane way of implementing it on small targets, and probably not even on large ones.

Indeed, for the smaller MCUs, it's a worse sin to have multiple similar code functions than printf being slow as customers can easily understand that printf can be and will be slow, due to its features.

Strictly speaking of printf() and its siblings, I am not even convinced whatsoever than using malloc() would speed anything up in the general case compared to using function pointers and so on. It may, on some specific platforms and with a specific set of test cases, but most likely not at all in the general case. The standard malloc() has undeterminate execution time and will be probably much slower on average than calling callback functions for each character.

That may prove marginally faster on VERY large formatted sequences, but most likely worse in general.

[richardman]

Programs are written by people who know how to program. The military and the avionic industry mandated the use of Ada in the 90s and on. People thought the era of "better software engineering" for such use would be upon us...

... and yet, most of those software are still and will be written using C and C++ after the mandatory Ada requirement was dropped. From what I understand, even though there has been decent, even "free" Ada compilers such as GNU Ada, it was and is difficult to attract enough talent.

So back to a case of supply and demand.

[legacy]

Programs are written by people who know how to program.

judging by all the effort we have to invest in our activities, let me say it may be a wrong assumption, and the C language doesn't put any barrier on this.

e.g. we considered an embedded FAT filesystem(1) given us by a third company, and it ended we had to spend three times the effort we would have invested in writing it from scratch 

(I had told it to my boss, but he didn't want to listen ...)

why? apart the big mistake made on the wrong assumption that we would have to use GCC(2) (which is extremely bad since it means the code is NOT portable), the authors were pretty ignorant about software design, they have completely ignored all the error propagation in their interfaces, and worse still, their poor modularity made the testing activity like a nightmare, which then usually costs more time to sort it out.

now, a lot of these concepts would have been mitigated with Ada, resulting in us to had spent less effort on it.

The military and the avionic industry mandated the use of Ada in the 90s and on. People thought the era of "better software engineering" for such use would be upon us... and yet, most of those software are still and will be written using C and C++

unfortunately, we can't use Ada for an OS, or for a BSP, we neither use the C++ for this, but it's irrelevant: the point is everything needs to be written in C requires a lot of effort from the engineering step the production step, and the effort we have to invest with C is several orders of magnitude higher than the effort we have to invest with Ada, which is ridiculous since what is written in Ada is usually of higher complexity.

From my experiences:

• C: low complexity design requires usually high effort (can't Imagine the effort for a high complexity design ... probably it's of the level of several chronic headaches)
• Ada: high complexity design usually requires low effort, and it's less error prone

I am one of the those have written the AFDX switch's firmware purely in Ada. It has already passed all the qualifications with a quarter of the effort requested by a C implementation, but ... for some strange reasons is not in production, and it's frustrating because for a reason, or for another, it really seems people really don't want to move on 


(1) it's for the navigation maps of tactical missions. It needs to burst-read large files from a USB pen drive. There is a dedicated board for this, with a dedicated interface to the tactical viewer on the pilot's helmet.
(2) we are usually with Diab and GreenHills.

[richardman]

Legacy, you just proved my point 

[legacy]

Legacy, you just proved my point 

maybe, but what really matters is that soon or late I will convince the project managers to accept my work in Ada, and it will be the end of the trouble we are having with the AFDX and ARINC stuff written in C!

That D-day, in the near future, will be remembered in every local calendar until the end of time, with at least one week of celebration and a boxed set of bottles of champagne offered to all the team, including all the secretaries, the guys, and girls working at the reception 

[westfw]

I am not convinced of the OP's claims

We never DID get a full description of which "string functions" were supposed to be using malloc(), did we?
OTOH, whether a given set of functions CAN be written without malloc() is only slightly comforting to the user faced with a library that DOES use malloc().   Yeah, lots of us are in favor of writing low level C code without resorting to "bloated vendor libraries", but we don't usually extend that to writing our own versions of the standard libc functions...

I am not even convinced whatsoever than using malloc() would speed anything up in the general case compared to using function pointers and so on.

Well, that's going to be pretty highly dependent on what is "underneath" putc(), isn't it?  I mean, putc() is defined to send characters to a file, and newlib (for example) has a whole very general stdio implementation underneath putc(), with quite a lot of overhead for each call to putc(), not counting any underlying "file" implementation (like an actual disk) that might be really "unhappy" being asked to do individual byte writes (probably there's another whole layer of malloc() for the file buffers.)I don't have any trouble believing that many cases are more efficient with an intermediate buffer somewhere in vsnprintf()
Even if you do "short-circuit" sprintf() via function pointers (or hard-coded check), vsnprintf() is still going to REFERENCE malloc(), in order to support asprintf() variants.  (I'm a little surprised that newlib doesn't have an "omit variations of functions that require malloc()" option, given MISRA and other aversions to malloc().(Ah, and doesn't "have libc functions that reference malloc(), but don't actually implement a working malloc because it's not allowed" sound like exactly the sort of thing that got Ariane's "AdaTran" code in trouble?)
Alas, newlib looks like the sort of code that's been so generalized and option-filled and modularized that it would be difficult for anyone to figure out by looking at the source code, unless they spend a lot of time looking at it :-(https://github.com/32bitmicro/newlib-nano-1.0/blob/master/newlib/libc/stdio/putc.c#L92