Author Topic: stdlib + MCU haters: std itoa() uses less resources!  (Read 17901 times)

0 Members and 1 Guest are viewing this topic.

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #125 on: June 06, 2023, 07:32:43 pm »
I just want to get to the bottom of things
Well, then, catch the brainwasher.
Play around with the number type "value", right in the function.
https://godbolt.org/z/son3EsKaK
Sorry, I do not understand.  Was my example code gibberish?

If you want to write just one signed and/or one unsigned conversion function, the best types to use are intmax_t and uintmax_t, because they represent the maximum width integer types the compiler supports.

Here is umaxtoa() and imaxtoa() with proper safety checks and comments describing the intent:
Code: [Select]
#include <stdint.h>

const char toa_basis[] = "0123456789abcdef";
char       toa_error[4] = "(?)";

// Note: sizeof(toa_basis) == strlen(toa_basis) + 1
#define  TOA_RADIX_MAX  (sizeof(toa_basis) - 1)

static char  internal_umaxtoa(uintmax_t val, char *dst, int radix)
{
    // Verify valid radix.
    if (radix < 2 || radix > (int)(TOA_RADIX_MAX))
        return 1;  // Error

    // Construct string starting with the least significant digit of 'val',
    // using a helper pointer 'end' since we need original value of 'dst' later.
    char *end = dst;
    do {
        // Current least significant digit in 'val' base 'radix'; then advance 'end'
        *(end++) = toa_basis[val % radix];
        // Drop next digit into the least significat digit place.
        val /= radix;
        // Repeat if 'val' is nonzero.
    } while (val);

    // Terminate the string.
    *end = '\0';

    // Now we have the numeric string in reverse order, from *dst to *(end-1).
    // For example, 1234 in decimal will yield "4321" with end==dst+4.
    // Reverse the string by swapping characters.
    while (dst < --end) {
        // We already decremented end to point to the char itself,
        // and need to swap '*dst' and '*end', and then increment 'dst'.
        const char  c = *dst;
        *(dst++) = *end;
        *end = c;
    }

    // Successful conversion.
    return 0;
}

// dst must have room for at least 1+ceil(log(val)/log(radix)) chars.
char *umaxtoa(uintmax_t val, char *dst, int radix)
{
    // If 'dst' is NULL, we can only error out.
    if (!dst)
        return toa_error;

    // Do the conversion using internal helper function.
    if (internal_umaxtoa(val, dst, radix)) {
        // Nonzero return value: Make 'dst' be an empty string, and return error.
        dst[0] = '\0'; 
        return toa_error;
    }

    // Success.  Return 'dst'.
    return dst;
}

// dst must have room for at least 2+ceil(log(abs(val))/log(radix)) chars.
char *imaxtoa(intmax_t val, char *dst, int radix)
{
    // If 'dst' is NULL, we can only error out.
    if (!dst)
        return toa_error;

    if (val < 0) {
        // Do the conversion using internal helper function, but skip the first char in 'dst'.
        if (internal_umaxtoa(-val, dst+1, radix)) {
            // Nonzero return value: Make 'dst' be an empty string, and return error.
            dst[0] = '\0';
            return toa_error;
        }
        // Success.  Set the negative sign, and return a pointer to it.
        dst[0] = '-';
        return dst;

    } else {
        // Do the conversion using internal helper function.
        if (internal_umaxtoa(val, dst, radix)) {
            // Nonzero return value: Make 'dst' be an empty string, and return error.
            dst[0] = '\0';
            return toa_error;
        }
        // Success.
        return dst;
    }
}
These have been tested to produce identical outputs to snprintf(buf, sizeof buf, "%ju", uval) for all 33-bit unsigned integers (and a few hundred million random 64-bit unsigned integers), and to snprintf(buf, sizeof buf, "%jd", ival) for all 33-bit signed integers (and a few hundred million random 64-bit signed integers), in base 10.

They intentionally defend against silly errors, like using a NULL buffer (perhaps it was dynamically allocated, but the allocation failed), as well as invalid radixes.  The only error you can do, is supply a buffer that is too small for the conversion.  We cannot check for the buffer size, since all we get is the pointer to its beginning.

Are they the fastest possible?  Absolutely not, because we must stay within the behaviour of the existing API, since we're using the xtoa() naming convention, which users expect to behave like they normally do.  Also, often the uintmax_t and intmax_t types are two or four machine words in size, and that makes the divide-and-modulo by radix much, much slower than necessary.  It would be useful to compare to UINT_MAX, and use internal_utoa() –– identical to internal_umaxtoa() except val is of type unsigned int ––, because it is either machine word in size, or the compiler implements arithmetic on it as efficiently as it can, making it a good candidate for the type used in typical conversions.
Alternatively, you could implement internal_u16toa() and/or internal_u32toa(); the former might be useful tradeoff on 8-bit microcontrollers.

Just limiting ourselves to the interfaces existing libraries provide –– including the above functions I have posted to this thread! –– may make the code slightly easier for others to maintain (in the sense that when they write code using these without consulting the documentation, the implementation makes typical bugs other than too-short buffer obvious in testing), but not true advancement.

What if we want something better?  Well, that was exactly what that thread I started way back was about, for microcontroller-type use cases ("constrained environments").  I prefer a completely different interface to itoa()/snprintf() myself.  The first question then is, what do we intend to use the conversions for: emitting to USB or UART, constructing longer strings into RAM buffers, or what?  The 'good' answers to that, when more closely examined, will likely yield completely different interfaces!  Especially if one considers stuff like _Generic() and macro expansion, which can help make an interface much easier to use.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #126 on: June 06, 2023, 07:55:16 pm »
Just limiting ourselves to the interfaces existing libraries provide –– including the above functions I have posted to this thread! –– may make the code slightly easier for others to maintain (in the sense that when they write code using these without consulting the documentation, the implementation makes typical bugs other than too-short buffer obvious in testing), but not true advancement.
True advancement doesn't come from C. A first step would be to move over to C++ with better string handling and needing less dealing with pointers. Even better would be Python (for larger targets) or Lua (for smaller targets) to reduce NRE costs. I expect prices of microcontrollers to drop sharply over the next years so memory size becomes even less of an issue.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline wek

  • Frequent Contributor
  • **
  • Posts: 495
  • Country: sk
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #127 on: June 06, 2023, 08:13:46 pm »
C sucks to begin with. So what.

C core language is still quite A) consistent, B) powerful. Standard library on the other hand is very inconsistent and limited in its abilities (so that programmer needs to re-write simple things over and over again), it looks like some people just cobbled something together in 1970's and then it was locked down with no changes allowed.

C core language was cobbled together in the 1970 in the same haphazard way than the libraries. If it feels consistent to you it's just because you've got accustomed to its idiosyncracies (it also helped a bit that it was thrown together basically by one person in a relatively short timespan).

"Powerful" is relative. Would it be powerful enough, most of what is in the standard libraries would be integral part of the language.

The single powerful feature of C is, that it does exist.

My point is not to bitch about C or the stdlib, though. My point is, that it is what it is, learn to live with it, according to your particular needs.

At the end of the day, any replacement would have its own idiosyncracies, too, and we'd just discuss those then.

And there's no one size fits all.

JW
 

Online MK14

  • Super Contributor
  • ***
  • Posts: 4539
  • Country: gb
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #128 on: June 06, 2023, 08:24:22 pm »
C core language is still quite A) consistent, B) powerful. Standard library on the other hand is very inconsistent and limited in its abilities (so that programmer needs to re-write simple things over and over again), it looks like some people just cobbled something together in 1970's and then it was locked down with no changes allowed.

C core language was cobbled together in the 1970 in the same haphazard way than the libraries. If it feels consistent to you it's just because you've got accustomed to its idiosyncracies (it also helped a bit that it was thrown together basically by one person in a relatively short timespan).

"Powerful" is relative. Would it be powerful enough, most of what is in the standard libraries would be integral part of the language.

The single powerful feature of C is, that it does exist.

My point is not to bitch about C or the stdlib, though. My point is, that it is what it is, learn to live with it, according to your particular needs.

At the end of the day, any replacement would have its own idiosyncracies, too, and we'd just discuss those then.

And there's no one size fits all.

JW

Well I asked my GCC compiler about this, and here is its reply:

Code: [Select]
#include <stdio.h>

int main() {
    printf("Oh, I see. You want aggression and annoyance, do you? Well, brace yourself for a dose of unapologetic bluntness.\n");
    printf("It's amusing how you dismissively refer to the C programming language as being cobbled together in a haphazard manner. Allow me to remind you that C was meticulously designed by the legendary Dennis Ritchie, a pioneer in the field of computer science. It was his brilliance that birthed this powerful language, which has withstood the test of time and continues to be a cornerstone in the world of programming.\n");
    printf("You seem to belittle the C standard library, claiming that if C were truly powerful, everything in the libraries would be integrated into the language itself. Let me enlighten you, my friend. The standard library is a separate entity precisely to maintain a modular and lightweight core language. It allows developers to add functionality as needed, keeping the language lean and adaptable. This approach has empowered programmers to create countless libraries and frameworks that extend the capabilities of C.\n");
    printf("Oh, and your assertion that the single powerful feature of C is its existence? That's just a feeble attempt at oversimplification. C's power lies in its efficiency, its low-level control, and its ability to interact closely with hardware. It's the language of choice for system programming, where performance and optimization are paramount. Its influence can be seen in countless software systems that underpin the modern world.\n");
    printf("Now, let's address your defeatist attitude. Instead of whining about C's idiosyncrasies, why not embrace them as the unique flavor that defines the language? Every programming language has its quirks, and C is no exception. It's about time you accept that fact and adapt to the realities of the language you're working with. If you can't handle it, maybe it's time to step aside and let the true C aficionados take the reins.\n");
    printf("And finally, your remark about there being no one-size-fits-all solution. Well, congratulations! You've stumbled upon an obvious truth. But don't use it as an excuse for mediocrity. The beauty of programming is in the diversity of languages and tools at our disposal. Instead of settling for complacency, we should strive to find the best fit for each unique problem and push the boundaries of what's possible.\n");
    printf("So, my exasperating friend, if you're going to criticize C, at least do so with some intellectual rigor and a genuine understanding of its strengths and legacy. Otherwise, your complaints come across as nothing more than the futile grumblings of an ignorant provocateur.\n");
   
    return 0;
}

With no help from ChatGPT  (cough cough).
 

Offline MMMarco

  • Regular Contributor
  • *
  • Posts: 69
  • Country: ch
  • Hobbyist. ⚠️ Opinionated
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #129 on: June 06, 2023, 08:31:45 pm »
With no help from ChatGPT  (cough cough).

I was about to say, this look suspiciously like the product of ChatGPT  :-DD
27 year old Software Engineer (mostly JavaScript) from Switzerland with a taste for low level stuff like electronics 😊

 
The following users thanked this post: MK14

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #130 on: June 06, 2023, 08:33:35 pm »
I think we got nice conclusions to the thread here. :-DD
 
The following users thanked this post: MK14

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #131 on: June 06, 2023, 09:13:53 pm »
C sucks to begin with. So what.

C core language is still quite A) consistent, B) powerful. Standard library on the other hand is very inconsistent and limited in its abilities (so that programmer needs to re-write simple things over and over again), it looks like some people just cobbled something together in 1970's and then it was locked down with no changes allowed.

C core language was cobbled together in the 1970 in the same haphazard way than the libraries. If it feels consistent to you it's just because you've got accustomed to its idiosyncracies (it also helped a bit that it was thrown together basically by one person in a relatively short timespan).

"Powerful" is relative. Would it be powerful enough, most of what is in the standard libraries would be integral part of the language.

The single powerful feature of C is, that it does exist.

My point is not to bitch about C or the stdlib, though. My point is, that it is what it is, learn to live with it, according to your particular needs.

At the end of the day, any replacement would have its own idiosyncracies, too, and we'd just discuss those then.
Not so long ago some people where hell-bound on programming microcontrollers in assembly  >:D History will repeat itself.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 
The following users thanked this post: MK14

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #132 on: June 06, 2023, 09:19:03 pm »
A first step would be to move over to C++ with better string handling and needing less dealing with pointers.
GCC does not support named address spaces for C++ (although Clang does), only for C, so you cannot really leverage Harvard architectures like AVRs without an unified memory model (like ATmega32U4 and AT90USB1286, my favourite AVRs).  This is exactly why e.g. Arduino copies many strings to RAM and does other similar inanities.

Now, on single address space microcontrollers like 32-bit ARM Cortex-M0/M0+/M1/M3/M4/M7, the situation is different.

However, the C++ standard leaves freestanding environment so implementation-defined, that one must either rely on the specific compiler implementation used, or rely mostly on the standard definitions for C freestanding environment.  The latter leads to the funky C/C++ freestanding environment many embedded developers use.  This excludes things like std::string, unless you use a library that implements them for you.  Nothing comes for free here.

Better look where you step, I say.
 
The following users thanked this post: MK14

Offline AVI-crak

  • Regular Contributor
  • *
  • Posts: 125
  • Country: ru
    • Rtos
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #133 on: June 06, 2023, 10:25:56 pm »
Sorry, I do not understand.  Was my example code gibberish?
Sorry friend, I can't understand why complex functions are needed at the lowest level.
I believe that simple things should work quickly, without unnecessary movements.
It is not possible to do it in one function - that means you need several different functions, the excess will not be used.
There is a wonderful abstraction on this topic: the more code is written, the less chance there is to use a single function standalone. As long as there are several paper clips in the cup, each of them can be lifted separately. But when there is a handful of paper clips, then one paper clip lifts almost everything. [ Specified attachment is not available ]
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #134 on: June 06, 2023, 10:32:54 pm »
Code bloat is something that is seemingly very hard to resist. It takes a specific mindset that very few people seem to have.
This is an almost lost cause in the software industry, except in very niche areas such as safety-critical stuff.

The moment you're asked to validate every piece of your design, including third-party code (aka SOUP), suddenly you're a lot more inclined to be lean.
 
The following users thanked this post: Siwastaja, elecdonia, MK14, uer166, MMMarco

Offline AVI-crak

  • Regular Contributor
  • *
  • Posts: 125
  • Country: ru
    • Rtos
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #135 on: June 06, 2023, 11:05:04 pm »
Code bloat is something that is seemingly very hard to resist.
The solution is very simple - do not mix all the paperclips in one cup.
Many different cups - for paper clips of different colors.
Place cups horizontally and vertically in layers.
Then one paperclip from the top cup can pull out only one long thread. Figuratively.

And by the way, we have a great example on this subject - "printf()". Just one line, and you have a dozen unnecessary functions dead weight. You can't remove it - the library was carefully compiled in advance - in one dense lump.
 

Offline MMMarco

  • Regular Contributor
  • *
  • Posts: 69
  • Country: ch
  • Hobbyist. ⚠️ Opinionated
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #136 on: June 06, 2023, 11:18:39 pm »
This is an almost lost cause in the software industry, except in very niche areas such as safety-critical stuff.

 :-+ that's the spirit! The less code, the less that can potentially go wrong.
27 year old Software Engineer (mostly JavaScript) from Switzerland with a taste for low level stuff like electronics 😊

 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #137 on: June 07, 2023, 03:48:57 am »
As others have already pointed out, it just happens to be that the C standard library API sucks to begin with.
C sucks to begin with. So what.

JW

I don't think C sucks. Sure there are some things that are borderline evil like unsafe functions like gets etc. but I much prefer C over C++.

"gets" is not C, it's library, and you don't have to use it.

C++ was pretty good in the 90s. Today it is insane. And C now has most of the good features that were only in C++ in the 90s.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #138 on: June 07, 2023, 04:00:03 am »
Just limiting ourselves to the interfaces existing libraries provide –– including the above functions I have posted to this thread! –– may make the code slightly easier for others to maintain (in the sense that when they write code using these without consulting the documentation, the implementation makes typical bugs other than too-short buffer obvious in testing), but not true advancement.
True advancement doesn't come from C. A first step would be to move over to C++ with better string handling and needing less dealing with pointers. Even better would be Python (for larger targets) or Lua (for smaller targets) to reduce NRE costs. I expect prices of microcontrollers to drop sharply over the next years so memory size becomes even less of an issue.

Disagree.

A CPU plus a couple of KB of RAM and ROM and 8 bits of GPIO is always the good enough for lots of things but low cost target.

With 8080 and 6800 it was $500. 6502 and z80 reduced it to $50. Several years back it was $1. Now it is $0.10. Maybe in future it will be $0.01.

There are many things for which it is worth using such a machine, even if something a little bigger that can run Python or Lua doesn't cost all that much more, and doing so needs asm or C.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #139 on: June 07, 2023, 07:38:06 am »
Nowadays lots of devices need connectivity which implies some form of security besides a lot of extra code. With C you need to add security to almost every function to avoid simple things like buffer overruns. If you can re-use a limited set of well tested functions in a structured way (for example by calling these from a scripted language to implement the functional behaviour of a product) you reduce the chance of such errors drastically.
« Last Edit: June 07, 2023, 07:50:32 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline MMMarco

  • Regular Contributor
  • *
  • Posts: 69
  • Country: ch
  • Hobbyist. ⚠️ Opinionated
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #140 on: June 07, 2023, 01:58:26 pm »
"gets" is not C, it's library, and you don't have to use it.

I don't know where you're going with this.

I never said it's part of the language.

However, it's de facto part of the language because it's included in the C standard library.

You can't just view the "language" on its own without its APIs ; a language without an API would be useless to most.

(A good language, has good syntax AND a good STANDARD library [that comes INCLUDED with the language]).

Sure, gets is probably not available on MCUs, but C isn't exclusively used for MCUs either.

So, I really don't get what your point is.

What I said is, C is fine but there are some things that get you in trouble real quick (like gets) it doesn't matter whether that's part of the language or not.

I could've made an entirely different example and it would still hold, for example sscanf which is what you would use to parse some parameters out of a string, can be unsafe (if you don't specify the buffer size) too.

On a hosted environment, you really don't want to re-write all the standard functions yourself - that would be silly.

So, it's technically not part of the language (syntax etc.) but practically speaking it is, because no-one in their right mind would re-write the standard library without a really strong case for it.

And that's what sucks about C, the standard library isn't really suited for modern applications, although with newer standard we're getting there.
« Last Edit: June 07, 2023, 02:10:40 pm by MMMarco »
27 year old Software Engineer (mostly JavaScript) from Switzerland with a taste for low level stuff like electronics 😊

 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #141 on: June 07, 2023, 02:59:55 pm »
So, [C standard library is] technically not part of the language (syntax etc.) but practically speaking it is, because no-one in their right mind would re-write the standard library without a really strong case for it.
No.  Many people assume it is, because they cannot specify anything better to replace it with.

Many software projects like the Linux kernel are developed purely in the freestanding C environment.
Not only is the standard library completely excluded, but they also interact with compiler developers quite heavily, exactly because they need the compilers to generate sensible code for certain patterns even when the C standard has left it implementation-defined, or worse, undefined behaviour.

Thus, it really is an error to think of the C standard library as an irremovable part of the C language.

You can only make the argument based on popularity or the consensus of a misinformed majority, but if we go that way, then logically we also have to agree with billions of flies that poop tastes good.
 

Offline MMMarco

  • Regular Contributor
  • *
  • Posts: 69
  • Country: ch
  • Hobbyist. ⚠️ Opinionated
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #142 on: June 07, 2023, 03:15:13 pm »
Many software projects like the Linux kernel are developed purely in the freestanding C environment.

What are you talking about? The Linux kernel is obviously an exception because it's literally the core of many operating systems.

I was talking about application level code.

I give you a relatively clear example:

If you told me that you re-wrote malloc (which is part of the standard C library) for a project (that runs on an modern OS) then I would tell you that this is stupid.

malloc is relatively good example because memory allocation is a complex problem (especially on modern OSes) and should be left to the operating system.

So if you're trying to make the argument that this would be a sane choice, I have to strongly disagree with you.

Thus, it really is an error to think of the C standard library as an irremovable part of the C language.

I agree with that, although I'm not sure why you would tell me, because I said "I never said it's part of the language." above.

« Last Edit: June 07, 2023, 03:25:31 pm by MMMarco »
27 year old Software Engineer (mostly JavaScript) from Switzerland with a taste for low level stuff like electronics 😊

 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #143 on: June 07, 2023, 04:12:47 pm »
I was talking about application level code.
Ah, okay.  I thought/assumed this was about code running on a microcontroller or similar embedded and constrained devices.

If you told me that you re-wrote malloc (which is part of the standard C library) for a project (that runs on an modern OS) then I would tell you that this is stupid.
I agree, if we are talking about writing our own malloc()/realloc()/free() functions.
But, I am not: I am talking about replacing such things with completely new functions.
I only showed the xxxtoa() implementations as a concession, to discuss the details of numeric-to-string conversion only; I do not use that interface in code I write myself.

When I talk about replacing the standard C library, I mean with something other than the interfaces defined for the standard C library in the hosted environment.  If I were to talk about writing my own implementation of the hosted environment standard C library, I'd use 'reimplement the standard C library'.

Thus, whenever we're talking about replacing the standard C library, I am specifically talking about something with completely different interfaces and functions, with function names different than those in the standard C library; and not just reimplementing a superset or subset of the standard C library interfaces.

Thus, it really is an error to think of the C standard library as an irremovable part of the C language.
I agree with that, although I'm not sure why you would tell me, because I said "I never said it's part of the language." above.
Me fail English then –– I do pretty often.  You did use the qualifier 'de facto', "in fact or in practice; in actual use or existence, regardless of official or legal status", and I disagree even with that qualifier, because a lot of the code that drives C language development, uses the freestanding environment completely without the standard library; and because a large part of the "stagnation of C" is actually stagnation of the standard C library!

I've mentioned it before, but the lack of strdup(), strndup(), getline(), getdelim(), asprintf(), vasprintf() etc. in the standard C library is utterly silly.  They are widely used, and definitely help write more robust real-world systems or application-level code.  (Some of them are in POSIX, some are GNU only, some have BSD equivalents.)
To me, this is proof of the stagnation of the standard library part of the C language.

My argument is that instead of replacing C with something completely new, we should look into replacing the standard C library first, with something better.
The design work for systems or application-level programming is huge, but for the microcontroller and embedded world, running on plain hardware without an OS, the set of library-provided features is much smaller, and quite feasible to tackle.

Doing that one will for sure discover some core C language properties that ought to be modified; but that I consider the next step, to be discussed when one has practical and useful proof to show how to do it better (in the form of a replacement for the C standard library, with completely different functions).

(I admit I have done quite a bit of experimentation and research into replacing the standard C library for Linux systems programming, but that stems from the fact that the Linux kernel provides interfaces that POSIX compatibility fights against; for example, per-thread credentials (user, group).)
« Last Edit: June 07, 2023, 04:18:00 pm by Nominal Animal »
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #144 on: June 07, 2023, 04:22:32 pm »
If you told me that you re-wrote malloc (which is part of the standard C library) for a project (that runs on an modern OS) then I would tell you that this is stupid.
I agree, if we are talking about writing our own malloc()/realloc()/free() functions.
But, I am not: I am talking about replacing such things with completely new functions.
I only showed the xxxtoa() implementations as a concession, to discuss the details of numeric-to-string conversion only; I do not use that interface in code I write myself.

When I talk about replacing the standard C library, I mean with something other than the interfaces defined for the standard C library in the hosted environment.  If I were to talk about writing my own implementation of the hosted environment standard C library, I'd use 'reimplement the standard C library'.
This begs the question: why doesn't such a library exist yet? For C++ there are various libraries / frameworks like Boost, Qt, Wxwidgets which are very popular that sit between applications and the standard libraries. Even if you don't need code to be portable, these libraries / frameworks are super usefull. I'm not aware of anything similar existing for plain C.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 8172
  • Country: fi
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #145 on: June 07, 2023, 04:53:23 pm »
I was talking about application level code.

Which is not obvious at all.

Maybe in 1997 the ratio of C userland programs vs. systems programming use was 1000:1 so that system projects like microcontroller/DSP programming or OS kernels looked like "exceptions". In $((2011+11+1)) however, use of C in userland application programming has gone down so much* that I wouldn't call freestanding C an "exception" anymore. Many many people only write that kind of embedded/freestanding C and use languages like Python for desktop apps. Maybe C++ for what they consider "serious" projects.

*) me being called weird for doing that

Quote
If you told me that you re-wrote malloc (which is part of the standard C library) for a project (that runs on an modern OS) then I would tell you that this is stupid.

You would be pretty short-sighted. It's not unusual at all to do a custom implementation of malloc() exactly because memory requirements in embedded projects are very different to those where usual malloc implementations (e.g. newlib) were written for. For example that recent case where a buggy... featured library malloc() in peter-h's project wasted half of the whole RAM.
« Last Edit: June 07, 2023, 04:57:40 pm by Siwastaja »
 
The following users thanked this post: AVI-crak

Online Siwastaja

  • Super Contributor
  • ***
  • Posts: 8172
  • Country: fi
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #146 on: June 07, 2023, 05:04:18 pm »
This begs the question: why doesn't such a library exist yet? For C++ there are various libraries / frameworks like Boost, Qt, Wxwidgets which are very popular that sit between applications and the standard libraries. Even if you don't need code to be portable, these libraries / frameworks are super usefull. I'm not aware of anything similar existing for plain C.

I think those are different. They can't replace the C++ standard library because they do not offer better/different interfaces to nearly all of the usual problems. Instead, they add to the standard library (e.g., threads, networking...). Additionally, these are quite complex beasts which are arguably bloated, difficult to use, with significant learning curve, some of which is inevitable because they are solving more complex problems.

Maybe Apache Portable Runtime (APR) is similar in C?
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #147 on: June 07, 2023, 05:52:33 pm »
This begs the question: why doesn't such a library exist yet? For C++ there are various libraries / frameworks like Boost, Qt, Wxwidgets which are very popular that sit between applications and the standard libraries. Even if you don't need code to be portable, these libraries / frameworks are super usefull. I'm not aware of anything similar existing for plain C.
I think those are different. They can't replace the C++ standard library because they do not offer better/different interfaces to nearly all of the usual problems. Instead, they add to the standard library (e.g., threads, networking...). Additionally, these are quite complex beasts which are arguably bloated, difficult to use, with significant learning curve, some of which is inevitable because they are solving more complex problems.
Not really. These frameworks actually DO offer easier to use interfaces for accessing files, dealing with strings, networking, threads, mutexes, semaphores, etc to replace the standard C/C++ library stuff. Ofcourse there is a learning curve for those who aren't familiar with the concepts yet but they make programming vastly easier, more time efficient and less buggy.
« Last Edit: June 07, 2023, 05:54:32 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline AVI-crak

  • Regular Contributor
  • *
  • Posts: 125
  • Country: ru
    • Rtos
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #148 on: June 07, 2023, 05:59:53 pm »
For the microcontroller environment, there are small "optimized" standard C libraries. Supplied pre-optimized and pre-compiled - you can't add anything new, you can't modify it, you can't remove a part. Libraries have a good weight - ready-made images for all architectures with declared support. Dozens of nested directories, hundreds of code variants of the same meaning.
How are they different - nothing!
I hate the "*.a" format, I know what's inside, it's terrible.

To download the source codes of the standard C libraries - you need to try very hard.
I had fun looking for changes in different versions of GCC. Changes can be saved on one notebook sheet!!! However, if you add updates to headers, copyrights, a date stamp, and other garbage, there will be a lot of changes. GCC is a graveyard of obsolete ideology.

A good reason to start over.
 

Offline MMMarco

  • Regular Contributor
  • *
  • Posts: 69
  • Country: ch
  • Hobbyist. ⚠️ Opinionated
Re: stdlib + MCU haters: std itoa() uses less resources!
« Reply #149 on: June 07, 2023, 06:28:49 pm »
Ah, okay.  I thought/assumed this was about code running on a microcontroller or similar embedded and constrained devices.

I can see why the folks here would assume that it's about microcontroller / embedded systems but (as we all know) C has more applications than just embedded.

For example, nginx, a very powerful and popular web server is written in C.

I agree, if we are talking about writing our own malloc()/realloc()/free() functions.
But, I am not: I am talking about replacing such things with completely new functions.
I only showed the xxxtoa() implementations as a concession, to discuss the details of numeric-to-string conversion only; I do not use that interface in code I write myself.

When I talk about replacing the standard C library, I mean with something other than the interfaces defined for the standard C library in the hosted environment.  If I were to talk about writing my own implementation of the hosted environment standard C library, I'd use 'reimplement the standard C library'.

I have the same viewpoint as you, it's almost impossible to ditch the "old" C "standard library" - something else has to replace it with a completely new interface. But then you might as well create a new language with a similar or same syntax as C. Rust looks promising, although I'm not sure how good the support for MCUs and other targets is.

I've mentioned it before, but the lack of strdup(), strndup(), getline(), getdelim(), asprintf(), vasprintf() etc. in the standard C library is utterly silly.  They are widely used, and definitely help write more robust real-world systems or application-level code.  (Some of them are in POSIX, some are GNU only, some have BSD equivalents.)
To me, this is proof of the stagnation of the standard library part of the C language.

Yeah. I think that's partly because POSIX has so many features to offer that if you have POSIX available, you don't care about the "C standard library" anymore.

And I think that only applies to "hosted environments" because many POSIX things do not apply to MCUs. [... meaning, C is clearly used outside the scope of MCUs/embedded] ; I would be surprised to learn that a MCU platform would support POSIX  ;D ;D
27 year old Software Engineer (mostly JavaScript) from Switzerland with a taste for low level stuff like electronics 😊

 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf