stdlib + MCU haters: std itoa() uses less resources!

[T3sl4co1l]

That's for sure.

Everything must have a good rationale in engineering.

Well.  A rationale.  It might not be a good one.  For example, neglecting to study a problem because of time or budget constraints, making heavy use of assumptions and partial models... or worse, applying rules of thumb because they're all one knows.

I suspect both of these apply in the current thread.  With "studying" replaced by familiarity with users and applications, since we're talking about rewriting a function (or sets of) that are very widely used.

Which, several people aren't being very careful specifying the scope of what they're doing here: a general purpose replacement for the library functions, or a very narrow purpose-made, size/speed optimized (or not, for the OP!) substitute?

Which, to clarify: my reaction for example is in regards to such a confusion.  If you're going to re-implement a function, and name it the same thing -- that strongly implies to readers that it should be a general-purpose substitute for the well-known version, and therefore any and all discrepancies against the library interface and behavior, are critical, source-breaking bugs.  (That is, a drop-in equivalent that can be compiled into any existing source with no problems.)

Tim

[Nominal Animal]

Nowadays I mandate the use of snprintf() for simplicity sake. It is known to exist and well documented.

Do your users actually use it correctly?  I.e., with the pattern
    int len = snprintf(buffer, sizeof buffer, FORMATTING, values...);
    if (len < 0 || len >= (int)sizeof buffer) {
        /* Fail! */
    }
or do they ignore its return value?  I bet the latter.  The len == (int)sizeof buffer case in particular leads to an unterminated buffer but with a complete string in it.

Unless you deviate from the snprintf() API, of course.

In most C libraries targeted for microcontrollers, printf() doesn't support printing floating point numbers for example. Are those implementations suddenly useless?

No, but I have seen dozens of Arduino developers being surprised as to why suddenly their sketch is significantly larger, or worse, no longer fit in their MCU.
They just cannot associate it with printf()'ing or snprintf()'ing floating-point numbers, often through some compiled-but-not-actually-used debugging output, for some reason.  (snprintf, unlike print, comes from avr-libc or newlibc in Arduino. print comes from the Arduino core Printable class.)

When you want to implement localization using the standard techniques, a printf() that fails to format e.g. %2$s (second additional parameter as a string) correctly (but happens to print %1$s correctly if it is the first conversion specifier, because it silently ignores the n$ part), can be even more aggravating than a printf() that balloons when you use it with floats.

I definitely do not see such a printf() as superior to a well-designed different new interface.

Often, when formatting floating-point numbers in a microcontroller, I am only interested in a specific range of values and at a limited precision.  For example, I might want to limit to range -99999999 to +99999999, with at most eight significant digits, with values exceeding the aforementioned range clamped to the limits.  (Possible values shown then includes +1234.5678 and -.87654321.)
With a just one additional macro and related formatting struct, I can add to the emit() interface I described earlier a LIMITED_FLOAT(value, limits) formatting method, where limits are a comma-separated list of (optional) structure initializers specifying the various limits, including things like whether to include integral zero for -1 < value < +1, whether to omit trailing fractional zero digits, and so on.  No parsing, as it is converted to a function call with the populated structure as a parameter (with unspecified fields zeros), so this uses minimal RAM and code.

(Formatting such range-limited floating-point values is much simpler/easier/more efficient than general floating-point formatting, especially if the maximum number of decimal digits is also specified.  Very little RAM is needed for the conversion, even when doing exact correct rounding.  It is the very large values, and nonzero values close to zero, that require more complex code and more RAM, when formatting floating-point numbers into decimal forms.)

Also, on Harvard architecture AVRs (like ATmega32U4 that I happen to like) the C99 emit() interface (or GCC and Clang _Generic, to be specific) can differentiate between __flash const char *, const char *, and char *, making it trivial to support immutable strings in Flash/ROM.  How would you do that with the printf() interface?  A new formatting specifier would work, but the compiler won't recognize it, leading to all sorts of hassles (see Linux kernel printk() in my earlier message).
The emit() interface does the right thing, and will give a compile-time error.  It won't silently map a __flash-qualified type to an otherwise matching non-__flash-qualified type.

[nctnico]

Which, to clarify: my reaction for example is in regards to such a confusion.  If you're going to re-implement a function, and name it the same thing -- that strongly implies to readers that it should be a general-purpose substitute for the well-known version, and therefore any and all discrepancies against the library interface and behavior, are critical, source-breaking bugs.  (That is, a drop-in equivalent that can be compiled into any existing source with no problems.)

The latter is never there in a resource constrained environment. And this is by design. If you use printf for a float without float support, it will print nothing. How that breaks existing code depends on what the resulting text is used for.

[nctnico]

Nowadays I mandate the use of snprintf() for simplicity sake. It is known to exist and well documented.

Do your users actually use it correctly?  I.e., with the pattern
    int len = snprintf(buffer, sizeof buffer, FORMATTING, values...);
    if (len < 0 || len >= (int)sizeof buffer) {
        /* Fail! */
    }
or do they ignore its return value?  I bet the latter.  The len == (int)sizeof buffer case in particular leads to an unterminated buffer but with a complete string in it.

No, snprintf always includes a terminating zero in the buffer unless the size of the buffer is 0. By design you make sure buffer is long enough for normal operation of the program. If that is not possible, rethink the design because it is likely prone to other errors. Preventing unterminated strings serves as a last resort barrier to prevent an error (memory corruption) from spreading like an oil stain that can't be traced to a source.

In most C libraries targeted for microcontrollers, printf() doesn't support printing floating point numbers for example. Are those implementations suddenly useless?

No, but I have seen dozens of Arduino developers being surprised as to why suddenly their sketch is significantly larger, or worse, no longer fit in their MCU.
They just cannot associate it with printf()'ing or snprintf()'ing floating-point numbers, often through some compiled-but-not-actually-used debugging output, for some reason.  (snprintf, unlike print, comes from avr-libc or newlibc in Arduino. print comes from the Arduino core Printable class.)

Chalk that up to learning curve. That is not the fault of using printf or a reason to invent something totally new. Use a smaller version of printf that is suitable for the job.

When you want to implement localization using the standard techniques, a printf() that fails to format e.g. %2$s (second additional parameter as a string) correctly (but happens to print %1$s correctly if it is the first conversion specifier, because it silently ignores the n$ part), can be even more aggravating than a printf() that balloons when you use it with floats.

You can always come up with cases that need a more elaborate printf function. In such cases use one that supports the required formatting at the expense of needing extra code space.

There is no one-size-fits all solution. Just use the type most appropriate. If it is warm outside wear a T-shirt, if it is cold outside wear a warm jacket. It is not more complicated than that.

[T3sl4co1l]

Which, to clarify: my reaction for example is in regards to such a confusion.  If you're going to re-implement a function, and name it the same thing -- that strongly implies to readers that it should be a general-purpose substitute for the well-known version, and therefore any and all discrepancies against the library interface and behavior, are critical, source-breaking bugs.  (That is, a drop-in equivalent that can be compiled into any existing source with no problems.)

The latter is never there in a resource constrained environment. And this is by design. If you use printf for a float without float support, it will print nothing. How that breaks existing code depends on what the resulting text is used for.

Which latter?  You only quoted one thing...

Is AVR "resource constrained"?  I don't know that I've ever not had float support when called upon, and I don't even know offhand how you can force disable its import.

If that's not constrained then I'm not sure why "never".  Perhaps definitionally, any platform with such support is not constrained? It's a usual example of a limited system though... *shrug*

Not that printf is particularly useful on an embedded system, but you can if you want to, and, a lot of people do.

Tim

[Nominal Animal]

Summary on the emit(ref, values...) interface, and why I believe it is superior to printf()/snprintf() in resource-constrained situations:

• err = emit(ref, value1, value2);
  is preprocessed to the equivalent of
  emit_init(ref);
  emit_type1(ref, value1);
  emit_type2(ref, value2);
  err = emit_end(ref);
  without any runtime parsing via C99 _Generic, thus reducing code size.
   
• The above also means that -ffunction-sections -Wl,--gc-sections will only keep the individual format-specific functions that are referred to in the code in the linked binary, reducing code size even further whenever possible.
   
• On Harvard architectures like AVRs, GCC and Clang include __flash qualifier in the _Generic type selection logic.
  This means that char *, const char *, __flash char *, and __flash const char * are four separate types that can be directed at four different functions.  _Generic will not map one to another silently, and will instead output an error message if the type is not supported.
  This allows developers to put strings in RAM or ROM/Flash, and trust the facility accesses it correctly (LPM instead of LP in AVRs).
  With printf(), this is problematic/difficult; even if you implement it, the compiler will not understand it correctly (as it is not standard formatting).
   
• Conversion of floating-point numbers to strings using default rules only requires a suitable formatting function.
   
• Conversion of floating-point numbers with formatting specification can be done using a struct wrapper.
  Basically, a macro like FORMATTED_FLOAT(value, ...spec...) constructs a structure that contains both the floating-point value and the spec parameters.  The formatting function is then of form
  emit_fmtfloat((struct formatted_float){ .value = (value), ...spec...);
  i.e. the structure is populated at the function call time, instead of taking valuable RAM.
   
• New formatting functions can be easily added.  Typically, it is just a matter of adding the type to the _Generic definition list.
   
• Minimal cognitive load, so both normal humans and 1337 coderz will get the interface right.
  There aren't any rules for ordinary output, and only special formatting of values or strings requires the use of a wrapper macro, hopefully well named.

[Nominal Animal]

Chalk that up to learning curve.

Why not require them to learn the better interface from the get go, when the existing one cannot be completely implemented or is problematic even if completely implemented?

Or does your opinion really boil down to "printf() is better, because in my opinion, printf() is better", as your above comment indicates?

[nctnico]

Which, to clarify: my reaction for example is in regards to such a confusion.  If you're going to re-implement a function, and name it the same thing -- that strongly implies to readers that it should be a general-purpose substitute for the well-known version, and therefore any and all discrepancies against the library interface and behavior, are critical, source-breaking bugs.  (That is, a drop-in equivalent that can be compiled into any existing source with no problems.)

The latter is never there in a resource constrained environment. And this is by design. If you use printf for a float without float support, it will print nothing. How that breaks existing code depends on what the resulting text is used for.

Which latter?  You only quoted one thing...

Is AVR "resource constrained"?  I don't know that I've ever not had float support when called upon, and I don't even know offhand how you can force disable its import.

ST's Cube IDE comes with float printing disabled out of the box. Early versions of MSPGCC (TI MSP30) have no float support at all. In my experience it is more common to not have float support in microcontrollers.

If that's not constrained then I'm not sure why "never".  Perhaps definitionally, any platform with such support is not constrained? It's a usual example of a limited system though... *shrug*

Not that printf is particularly useful on an embedded system, but you can if you want to, and, a lot of people do.

I'm using printf (in various incarnations) in all my embedded projects as part as a cli that can be used to request status information and / or change settings. This has been proven to be super useful as it greatly simplifies checking & debugging systems that are running in the field. Printf not being useful in embedded platforms is one of the dogmas that should have been killed a long time ago.

For my most recent project the cli even works over bluetooth using a standard phone app. Customer is super happy because it allowed them to evaluate some aspects of their new product quickly without needing their phone app to be ready and / or setting up a development environment to make changes in the software.

[nctnico]

Chalk that up to learning curve.

Why not require them to learn the better interface from the get go, when the existing one cannot be completely implemented or is problematic even if completely implemented?

Or does your opinion really boil down to "printf() is better, because in my opinion, printf() is better", as your above comment indicates?

IMO: If there is already something better available from the platform library use that, if not, change the existing printf to a more suitable one. Either way you are using something that is at least standarised to some level and has online information on how to get things done.

[Siwastaja]

In my experience it is more common to not have float support in microcontrollers.

Times are changing. Maybe 15 years ago FPU in an MCU was a weird flagship thing, but today's $1 microcontroller is some Cortex-M4 with at least single precision FPU, so float arithmetics itself comes with no extra flash use and good runtime performance. Yet these things might have as little as 64K of ROM. It is wasteful not to use floats when the hardware is available, for workloads that benefit from using them. Then one probably wants to print them, too, but sacrificing one third of program memory for printf() is a no-go. So this is a very real modern-day issue.

[SiliconWizard]

All of this for initially a couple-line function that the OP didn't complain having to write, but just noted that it allegedly would take up more resources than the (non-standard) itoa() provided by newlib, which, as someone quickly pointed out, was a flawed analysis anyway.

Oh well.

As to T3sl4co1l's note, which was IMO not the core of the discussion but which concerned him, certainly I would strongly recommend not to re-use an existing "standard" function name for something you implement yourself, especially if it's not 100% compliant with the original. Use another name. Certainly.

Now as everything else in engineering, you have to adapt rules to the circumstances. If you use a custom implementation of, say, printf() in some embedded project  - not a gigantic desktop software project with tens of contributors - that is not fully compliant with the standard printf(), that's not a big deal. I still tend to favor choosing other names - and possibly other interfaces than printf(), as some have already suggested in other threads in the past - but otherwise that is no big deal.

And finally, newlib, even in its 'nano' variant, is still a bit of a dog. picolibc is an overall better solution at least for small embedded targets.
And that said, nothing forces you to ever use a single std lib function in some MCU project. The basic memset(), memcpy() and memmove() are directly inlined by compilers these days, as to the rest, I would not miss a lot of the standard functions.

[Siwastaja]

All of this for initially a couple-line function that the OP didn't complain having to write, but just noted that it allegedly would take up more resources than the (non-standard) itoa() provided by newlib, which, as someone quickly pointed out, was a flawed analysis anyway.

Hilarious, isn't it?
>I need a small itoa() for a device with 16KB of flash and already use LTO to get size down, so I experimented with different itoa() implementations for further savings
<There really is no reason to avoid using printf and it's cousins, regards, nctnico

[nctnico]

Complaining about a thread going offtopic  While you are at it, go and check all the other threads that went offtopic at some point.

[nctnico]

Now as everything else in engineering, you have to adapt rules to the circumstances. If you use a custom implementation of, say, printf() in some embedded project  - not a gigantic desktop software project with tens of contributors - that is not fully compliant with the standard printf(), that's not a big deal. I still tend to favor choosing other names - and possibly other interfaces than printf(), as some have already suggested in other threads in the past - but otherwise that is no big deal.

I have tried that but since I share code over a wide variety of platforms (including testing on a PC), having different printfs (as an example, there are many other functions) becomes a big mess quickly with code becoming hard to re-use. Similar for having cut down standard C libraries that provide the same names. This is all with the goal of being able to keep using the same source code. Imagine needing ifdefs in all Linux software to make it compile using uclibc? It would defeat the purpose of uclibc. The same for having a Posix / BSD style socket layer on embedded targets which typically only support a subset which is good enough for the majority of the use cases.

[brucehoult]

Times are changing. Maybe 15 years ago FPU in an MCU was a weird flagship thing, but today's $1 microcontroller is some Cortex-M4 with at least single precision FPU

But now the cost-sensitive are using $0.10 microcontrollers (whether WCH RISC-V RV32EC or Puya CM0+), with 2 or 3 KB RAM and 16k flash.

Even if you want to use FP, software FP is fast enough (~5µs for + - *) on an AVR for many purposes. It's mostly just the flash space is a problem on the smaller devices.

[Siwastaja]

Complaining about a thread going offtopic

Not at all, offtopic is fine to me. I find it just hilarious you give an off-topic statement of quite absolute nature ("really no reason"), the proof to show it false being right above your very nose.

[kgavionics]

In my case I don't use the bellow function to convert an Ultra long to string, not because of the size, or the bloat, but the fact that its output string is led by zeros, which I replace with spaces and each time the function is called it overwrites the old string, which is good, because I don't have to worry about clearing the old string, For instance with prinft function if you have a counter that counts from 0 to 255 and rolls over to 0, the hundred will be overwritten, but not the tens and ones, and you have to clear them manually by writing spaces after the counts rolls over!

Code: [Select]

void UlToStr(char* s, unsigned long bin, unsigned char n)
{
uint8_t temp=n;

    s += n;
    *s = '\0';

    while (n--)
    {
        *--s = (bin % 10) + '0';
        bin /= 10;
    }
n=temp;
   for (uint8_t i = 0;i < n-1;i++)
    {
        if (s[i] == '0')
     
s[i] = ' ';
else  break;

    }


}


[SiliconWizard]

Nowadays I mandate the use of snprintf() for simplicity sake. It is known to exist and well documented.

Do your users actually use it correctly?  I.e., with the pattern
    int len = snprintf(buffer, sizeof buffer, FORMATTING, values...);
    if (len < 0 || len >= (int)sizeof buffer) {
        /* Fail! */
    }
or do they ignore its return value?  I bet the latter.  The len == (int)sizeof buffer case in particular leads to an unterminated buffer but with a complete string in it.

When a truncated string would have no consequence (such as for some data logging or such), I don't test the return value and systematically set the last char of the buffer to zero right after the call to snprintf().
Otherwise, I check the return value and act accordingly.

Doing neither is (almost) as bad as just using sprintf().

[Siwastaja]

While the C standard, at least since C99 clearly says snprintf always must create zero-terminated string for any n > 0, googling about this reveals that the world is full of broken implementations hanging around, so adding buf[sizeof buf -1] = 0; after the call is pretty common practice; this is what I have been doing just in case, too. It's a pretty serious failure and a manifest how short-sighted some library designers / writers can be. One more reason I don't see much value in the usual "libraries are always much better written that what you can do by yourself" argument.

[brucehoult]

One more reason I don't see much value in the usual "libraries are always much better written that what you can do by yourself" argument.

Well, sure, I'm confident I can write any given library function better that the one in the library -- especially since I know in which directions my usage is or is not going to stress it -- but I don't have time to rewrite ALL of them.

[AVI-crak]

but I don't have time to rewrite ALL of them.

Rewriting everything is a very bad idea. Difficult free work, with the loss of your time.
Even if successful, there will be more enemies than fans.

Besides - this world is already used to bad terrible decisions.
We all listen to modern music, which is guaranteed to make a 17th century man lose his mind.
  We watch films from an endless kaleidoscope of three-second events (something new every three seconds). At the same time, I personally have a feeling that such films are made exclusively for people with attention deficit disorder.
Even our food has more ingredients to maintain its appearance than the weight of the food itself (packaging + chemistry).

This is the very ideal case when you can abandon the old version, and really write something new. No matter how many enemies and admirers there are. If you are comfortable, then the opinion of others is no longer important.

[Nominal Animal]

While the C standard, at least since C99 clearly says snprintf always must create zero-terminated string for any n > 0, googling about this reveals that the world is full of broken implementations hanging around, so adding buf[sizeof buf -1] = 0; after the call is pretty common practice; this is what I have been doing just in case, too. It's a pretty serious failure and a manifest how short-sighted some library designers / writers can be. One more reason I don't see much value in the usual "libraries are always much better written that what you can do by yourself" argument.

Exactly.  It also says that (for n > 0) "Thus, the null-terminated output has been completely written if and only if the returned value is nonnegative and less than n".  If the return value is zero, then the source string was an empty string (either formatting string empty, or consisting of conversions that produce no output).  If the return value is n, then a standards-compliant library has truncated the output by one character, but many (historical) buggy implementations left the array contents complete but unterminated.

(I thought glibc 2.0 era did that, but it actually returned -1 if the output was truncated, according to the man page.)

Funnily enough, the standard does not require strncpy() use the last character for the nul (stating that if the first n chars of source does not include a nul, then the destination buffer won't have a nul either).  Which lead to BSD strlcpy() and strlcat().  The wording of the standard has caused some strncpy() implementations to fill the rest of the buffers with zero (some still claim it does), which significantly slows the implementations down without any benefits.

(Kernels and other privilege boundary crossing security-sensitive code cannot rely on just filling the rest of the buffer with zeroes; they also need to worry about things like structure padding containing sensitive values usable in side channel attacks.  Only copying the individual fields (not full structures) and strings to userspace-allocated (allocated and initialized by the less privileged side) structures and buffers seems to be the most robust approach.)

Also, strncpy(dest, dest+i, n) is Undefined Behaviour for 0 <= i <= n when n > 0.  Some implementations deal with it sensibly, others do not.

Most ridiculously, neither strdup() nor strndup() are in standard C yet!  (They're in POSIX, though. asprintf() and vasprintf() (for <stdargs.h> va_list argument lists), for dynamically allocating a target buffer to print into, is still a GNU extension.  However, this is outside the scope here, because it is not useful in memory-constrained embedded/microcontroller environments, only really in fully hosted environments with lots of address space (so memory fragmentation due to heavy dynamic memory use is not a problem).

Personally, I find the Surprise in difference in behaviour of snprintf() and strncpy() a bug in the standard.
Weight(Principle of Least Surprise) > Weight(KISS), in my opinion.

[MMMarco]

Just had to show this to the old "I never use std libs in MCU" dogs  , showing it's not always the case.
(Sure enough, the flash usage will blast off when using any std print function)
Custom itoa, taken form here (Can it get more simple than this?):

Code: [Select]

char* _itoa(int val, int base){
    static char buf[32];
    int i=30;
    for(; val && i ; --i, val /= base)
      buf[i] = "0123456789ABCDEF"[val % base];
    return &buf[i+1];
  }

End result was standard itoa using less resources overall  . In custom itoa, sdtlib is not used at all.

I don't know if this has been already said on this thread (it's a long thread) but "short and efficient looking code" does not automatically translate to "short and efficient assembly / machine code".

[SiliconWizard]

Personally, I find the Surprise in difference in behaviour of snprintf() and strncpy() a bug in the standard.

Yes, and this probably one reason some implementations of snprintf() are buggy and do not ensure a properly terminated string in case of truncation. Confusing.

String functions in the C std are a disaster for the most part anyway. They are fine for anything non-critical, but I would never use them in any critical code.

As to the Principle of Least Surprise, this is exactly one good reason for writing stuff yourself. What you write yourself has the least chance of surprising you. (Or if it does, you have a bigger problem.)

[nctnico]

Only problem is that a) quite a few people suffer from Dunning-Kruger and b) most people are blind to their own mistakes.  IOW: if you want to be really sure your software works, have it looked over & tested by somebody else. Otherwise it really is better to find a good library and check the source code to make sure. Then at least two people have looked at the code reducing the chances of serious mistakes.