Author Topic: GCC: How to get a vsprintf from sprintf?  (Read 2340 times)

0 Members and 1 Guest are viewing this topic.

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
GCC: How to get a vsprintf from sprintf?
« on: July 29, 2022, 11:23:43 am »
I have a "printf" library which I have successfully used to replace the source-less ST Cube supplied stuff (which uses the heap, etc) but upon analysing .map file modules I have discovered that I have 1 place where a vsprintf is being called so I need to create that.

A bit of context is here
https://www.eevblog.com/forum/programming/best-thread-safe-printf-and-why-does-printf-need-the-heap-for-f-etc/

and the library I have installed is
https://github.com/MaJerle/lwprintf
which unfortunately doesn't have it.

It also doesn't have

svfprintf
svfiprintf
fiprintf
vfiprintf

but nothing in my project is calling any of those AFAICT, and I struggle to find out what exactly what most of them do. However vsprintf is being used.

EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.
« Last Edit: July 29, 2022, 02:45:18 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: GCC: How to get a vsprintf from sprintf?
« Reply #1 on: July 29, 2022, 07:12:46 pm »
EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.

Not just in "embedded work". One should avoid sprintf() as well, and any function that's supposed to fill a buffer and that doesn't take the max size as a parameter in general.

Then I'm not sure what you mean by "So maybe this is a better way to go to get vsprintf". Use vsnprintf () instead (and make sure to pass the right max size depending on context). That will require refactoring, but it's definitely eons better.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #2 on: July 29, 2022, 07:42:59 pm »
Sure; the thing is I am putting together something which will be programmed by others, and they may want it.

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline TheCalligrapher

  • Regular Contributor
  • *
  • Posts: 151
  • Country: us
Re: GCC: How to get a vsprintf from sprintf?
« Reply #3 on: July 29, 2022, 07:46:02 pm »
EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.

What is so special about embedded to claim that "one should never use the latter" in "embedded work" specifically?

If you hold such a radical stance towards `vsprintf`, it should probably apply to any kind of work, shouldn't it?
« Last Edit: July 29, 2022, 07:47:43 pm by TheCalligrapher »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #4 on: July 30, 2022, 08:09:15 am »
I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: GCC: How to get a vsprintf from sprintf?
« Reply #5 on: July 30, 2022, 08:29:54 am »
I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?

I don't follow your comments. vsnprintf is identical to vsprintf, except vsnprintf is safer. If you need vsprintf, simply use vsnprintf instead.

On the other hand, vsprintf and sprintf have different argument lists. They are not replaceable equivalents. But neither of them should be used when vsnprintf and snprintf exist.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #6 on: July 30, 2022, 08:59:03 am »
I was just after help on my question :)

I also didn't want to get into complications, but the explanation is in the 1st link in my 1st post. I currently have vsprintf but it isn't thread-safe because it lives in the ST-supplied libc.a standard C library which uses (probably) statics and (certainly) the heap. And whether you like it or not, people do use sprintf and vsprintf, and if they do they will get all sorts of funny problems. My sprintf is however thread-safe because I have a nice library for most of these, but it doesn't have vsprintf.

So I have two options: create a vsprintf from one of the other (thread-safe) functions, or undefine it so people can't use it. I tried removing it from the ST lib (objcopy strip symbol option) but that doesn't work because that lib (compiled, no source) has vsprintf defined (it has the code for it) and objcopy doesn't allow the symbol removal in such a case.

Is it hard to get a vsprintf from sprintf, with a function, or a macro?

For example, I have the following from that open source printf.c mentioned earlier. A "v" function calls va_start and va_end around the printf.

Code: [Select]
int printf_(const char* format, ...)
{
  va_list va;
  va_start(va, format);
  char buffer[1];
  const int ret = _vsnprintf(_out_char, buffer, (size_t)-1, format, va);
  va_end(va);
  return ret;
}


int sprintf_(char* buffer, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const int ret = _vsnprintf(_out_buffer, buffer, (size_t)-1, format, va);
  va_end(va);
  return ret;
}


int snprintf_(char* buffer, size_t count, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const int ret = _vsnprintf(_out_buffer, buffer, count, format, va);
  va_end(va);
  return ret;
}


int vprintf_(const char* format, va_list va)
{
  char buffer[1];
  return _vsnprintf(_out_char, buffer, (size_t)-1, format, va);
}


int vsnprintf_(char* buffer, size_t count, const char* format, va_list va)
{
  return _vsnprintf(_out_buffer, buffer, count, format, va);
}


int fctprintf(void (*out)(char character, void* arg), void* arg, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const out_fct_wrap_type out_fct_wrap = { out, arg };
  const int ret = _vsnprintf(_out_fct, (char*)(uintptr_t)&out_fct_wrap, (size_t)-1, format, va);
  va_end(va);
  return ret;
}
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 4392
  • Country: dk
Re: GCC: How to get a vsprintf from sprintf?
« Reply #7 on: July 30, 2022, 09:08:29 am »
I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?

I use https://github.com/eyalroz/printf  it implements vsprintf as:

Code: [Select]
int vsprintf_(char* s, const char* format, va_list arg)
{
  return vsnprintf_(s, PRINTF_MAX_POSSIBLE_BUFFER_SIZE, format, arg);
}
 
The following users thanked this post: peter-h

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: GCC: How to get a vsprintf from sprintf?
« Reply #8 on: July 30, 2022, 09:13:46 am »
I was just after help on my question :)

I also didn't want to get into complications, but the explanation is in the 1st link in my 1st post. I currently have vsprintf but it isn't thread-safe because it lives in the ST-supplied libc.a standard C library which uses (probably) statics and (certainly) the heap. And whether you like it or not, people do use sprintf and vsprintf, and if they do they will get all sorts of funny problems. My sprintf is however thread-safe because I have a nice library for most of these, but it doesn't have vsprintf.

So I have two options: create a vsprintf from one of the other (thread-safe) functions, or undefine it so people can't use it. I tried removing it from the ST lib (objcopy strip symbol option) but that doesn't work because that lib (compiled, no source) has vsprintf defined (it has the code for it) and objcopy doesn't allow the symbol removal in such a case.

Is it hard to get a vsprintf from sprintf, with a function, or a macro?

For example, I have the following from that open source printf.c mentioned earlier. A "v" function calls va_start and va_end around the printf.
Code: [Select]
...

I'm just really not able to understand what you are getting hung up on.

In your first post you said you have vsnprintf available.

In the code you listed above, vsnprintf is there.

Why can you not simply use vsnprintf?

For example, vsprintf looks like this:
Code: [Select]
int vsprintf (char * s, const char * format, va_list arg );
While vsnprintf is like this:
Code: [Select]
int vsnprintf (char * s, size_t n, const char * format, va_list arg );
The only difference is the size of the output buffer, n. If you don't know the size of the buffer, just put in a big arbitrary number for n, like 10000 or something. If you complain that this is unreasonable and it may cause a buffer overrun, well this is exactly why you shouldn't be using the unsafe version in the first place.

Also, if you don't know the size of the buffer, the code is already in big trouble.
« Last Edit: July 30, 2022, 09:15:38 am by IanB »
 
The following users thanked this post: Siwastaja

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #9 on: July 30, 2022, 09:49:35 am »
langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: GCC: How to get a vsprintf from sprintf?
« Reply #10 on: July 30, 2022, 06:59:04 pm »
langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

Of course. If you pass a very large value to it, then it essentially becomes equivalent.

I would still personally deprecate the use of vsprintf() in your system and warn your users about it. They won't hold your willingness to make the code more secure against you. As long as you help them a little:

One way of doing that without angering them to no end is to use a macro that, when defined, will allow the use of deprecated functions. (Alternatively, you can invert the semantics in order for your users not to have to take extra steps, and make it so that when a specific macro is defined, it will prevent the code from compiling if using some of the deprecated functions. Then explain in your docs in a paragraph the rationale and how to handle it.)
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 4392
  • Country: dk
Re: GCC: How to get a vsprintf from sprintf?
« Reply #11 on: July 30, 2022, 07:23:28 pm »
langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

might want to do:

Code: [Select]

__attribute__((deprecated("Use vsnprintf instead"))) int vsprintf(char* s, const char* format, va_list arg)
{
  return vsnprintf(s, PRINTF_MAX_POSSIBLE_BUFFER_SIZE, format, arg);
}
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #12 on: July 30, 2022, 09:22:24 pm »
Thanks - that's an interesting way to do a compile-time warning.

Re sprintf, I have some code which is full of these (ex ST, in fact ex FreeRTOS actually! - it is used to generate their dynamic task listing) and what they are doing is emitting fairly short strings of say 20 bytes but a lot of them, into a buffer whose size has been experimentally determined as needing ~1200 bytes, so I set it at 1500.

One can get the #bytes each snprintf has output (or tried to output, in case of truncation) but people tend to be too lazy to be adding up these bits as they go along.

And then if the buffer is on the stack (by far the easiest way, because it gets chucked out upon exit) its size is fixed anyway.

One could code it but as I say people tend to be lazy.

One couldd also prefill the buffer with something and before each sprintf check that value is still there at buffer[sizeof(buffer)-20], say.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6796
  • Country: va
Re: GCC: How to get a vsprintf from sprintf?
« Reply #13 on: August 01, 2022, 11:36:32 pm »
Quote
One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: GCC: How to get a vsprintf from sprintf?
« Reply #14 on: August 02, 2022, 12:12:05 am »
Quote
One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.

Yeah. As I've understood, peter-h designs products the customers of which can customize the firmware, so that would explain his position.

While you can choose who you work with (if you're the boss at least), it's a lot harder to choose your customers. Or that might mean giving up on a number of them, or spending endless hours on support. So I feel the pain. (Peter can correct me if I didn't get his constraints properly though...)
 

Online tellurium

  • Regular Contributor
  • *
  • Posts: 226
  • Country: ua
Re: GCC: How to get a vsprintf from sprintf?
« Reply #15 on: August 02, 2022, 05:33:58 am »
Another possibility would be to add some small scripting engine to the product, import a bunch of performance-critical functions from C, and provide customers with a scripting API instead of a C API.

My company did that a lot, we had customers that built their products entirely on a scripting engine.
Open source embedded network library https://mongoose.ws
TCP/IP stack + TLS1.3 + HTTP/WebSocket/MQTT in a single file
 

Online Berni

  • Super Contributor
  • ***
  • Posts: 4922
  • Country: si
Re: GCC: How to get a vsprintf from sprintf?
« Reply #16 on: August 02, 2022, 06:13:11 am »
langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

Of course. If you pass a very large value to it, then it essentially becomes equivalent.

I would still personally deprecate the use of vsprintf() in your system and warn your users about it. They won't hold your willingness to make the code more secure against you. As long as you help them a little:

One way of doing that without angering them to no end is to use a macro that, when defined, will allow the use of deprecated functions. (Alternatively, you can invert the semantics in order for your users not to have to take extra steps, and make it so that when a specific macro is defined, it will prevent the code from compiling if using some of the deprecated functions. Then explain in your docs in a paragraph the rationale and how to handle it.)

Yep this macro is a good solution in my opinion too.

If they try to use the unsafe versions they will get a long descriptive error of why this is bad. But at the same time they have the option of adding a macro define that turns it into a regular warning and lets the code compile anyway by passing the unsafe versions over to the safe versions by telling them they have an effectively infinite buffer size.

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output. It becomes a problem when you start using %s that could potentially make the string very long, especially if the data is coming from an outside source. That's where the safe versions should absolutely be used. Those things are the source of most security exploits.
 

Offline IanB

  • Super Contributor
  • ***
  • Posts: 11790
  • Country: us
Re: GCC: How to get a vsprintf from sprintf?
« Reply #17 on: August 02, 2022, 07:36:12 am »
That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output.

On the other hand, if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.
 

Online Berni

  • Super Contributor
  • ***
  • Posts: 4922
  • Country: si
Re: GCC: How to get a vsprintf from sprintf?
« Reply #18 on: August 02, 2022, 08:00:54 am »
That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output.

On the other hand, if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.

Yes but that means you are either using %s to join strings, or you are moving your pointer down along the buffer. In that case you no longer have a guarantee of having enough space.

But in general on MCUs you don't tend to build very long strings. It is common practice to just build the part of the string you need and imidietly flush it out to UART, on a display, onto a SD card..etc since you only have a few kilobytes of RAM to work with anyway.
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 6796
  • Country: va
Re: GCC: How to get a vsprintf from sprintf?
« Reply #19 on: August 02, 2022, 09:25:02 am »
Quote
One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.

Yeah. As I've understood, peter-h designs products the customers of which can customize the firmware, so that would explain his position.

Fair enough. I'd still want to offload technical support, though :)
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14297
  • Country: fr
Re: GCC: How to get a vsprintf from sprintf?
« Reply #20 on: August 02, 2022, 06:13:07 pm »
That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break.

Sure, but it still relies on making assumptions instead of having a safeguard. I have surely used sprintf() a lot in the past, as many of us, I'm sure. I don't anymore.

It doesn't cost you any more work to use snprintf(), a fixed-size buffer and passing the size of said buffer to snprintf. The runtime overhead is negligible. In the possibly rare event of the string being truncated because you didn't choose an appropriate buffer size, or because there was an unexpected input, then it will just truncate the resulting string instead of crashing. You can still check the return value to see if it fits or not.

This is part of a more general approach of doing (almost systematic) "input validation", which I'm a strong proponent of.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 26755
  • Country: nl
    • NCT Developments
Re: GCC: How to get a vsprintf from sprintf?
« Reply #21 on: August 02, 2022, 06:29:45 pm »
That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break.

Sure, but it still relies on making assumptions instead of having a safeguard. I have surely used sprintf() a lot in the past, as many of us, I'm sure. I don't anymore.

It doesn't cost you any more work to use snprintf(), a fixed-size buffer and passing the size of said buffer to snprintf. The runtime overhead is negligible. In the possibly rare event of the string being truncated because you didn't choose an appropriate buffer size, or because there was an unexpected input, then it will just truncate the resulting string instead of crashing. You can still check the return value to see if it fits or not.

This is part of a more general approach of doing (almost systematic) "input validation", which I'm a strong proponent of.
I agree. Especially if customers (read: noobs) are going to write code, only make snprintf() available. On embedded systems the buffer is more likely to be too small rather than too large causing all kinds of weird behaviours / hard to catch bugs due to overwriting other data. In the end it won't help confidence in the product might raise all kinds of support questions.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3671
  • Country: gb
  • Doing electronics since the 1960s...
Re: GCC: How to get a vsprintf from sprintf?
« Reply #22 on: August 06, 2022, 06:41:52 am »
Quote
Not sure I would want to be associated with any product programmed by that sort of developer.

Thank you for the compliment but you don't have to be :) In this case the customer will write his own module and it is unlikely you will end up using it.

Quote
Peter can correct me if I didn't get his constraints properly though...)

In this case that's correct, but actually a vast amount of ST supplied code uses just an sprintf... Looking around, snprintf is rarely used.

Quote
we had customers that built their products entirely on a scripting engine.

A lot of work and a lot of RAM. Also a lot of people know C well enough.

Quote
using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output. It becomes a problem when you start using %s that could potentially make the string very long, especially if the data is coming from an outside source. That's where the safe versions should absolutely be used. Those things are the source of most security exploits.

Exactly.

Quote
if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.

Recently I dealt with such a case by planting a marker 100 bytes before the end of a 1500 byte buffer, and checking it before each "tiny sprintf". That code produced a long table



If the marker is gone, you get a ** BUFFER OVERFLOW ** in there. Most of the values come from deep down in somebody else's (FreeRTOS actually) code.

Quote
But in general on MCUs you don't tend to build very long strings. It is common practice to just build the part of the string you need and imidietly flush it out to UART, on a display, onto a SD card..etc since you only have a few kilobytes of RAM to work with anyway.

Exactly.



« Last Edit: August 06, 2022, 06:46:36 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf