GCC: How to get a vsprintf from sprintf?

[peter-h]

I have a "printf" library which I have successfully used to replace the source-less ST Cube supplied stuff (which uses the heap, etc) but upon analysing .map file modules I have discovered that I have 1 place where a vsprintf is being called so I need to create that.

A bit of context is here
https://www.eevblog.com/forum/programming/best-thread-safe-printf-and-why-does-printf-need-the-heap-for-f-etc/

and the library I have installed is
https://github.com/MaJerle/lwprintf
which unfortunately doesn't have it.

It also doesn't have

svfprintf
svfiprintf
fiprintf
vfiprintf

but nothing in my project is calling any of those AFAICT, and I struggle to find out what exactly what most of them do. However vsprintf is being used.

EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.

[SiliconWizard]

EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.

Not just in "embedded work". One should avoid sprintf() as well, and any function that's supposed to fill a buffer and that doesn't take the max size as a parameter in general.

Then I'm not sure what you mean by "So maybe this is a better way to go to get vsprintf". Use vsnprintf () instead (and make sure to pass the right max size depending on context). That will require refactoring, but it's definitely eons better.

[peter-h]

Sure; the thing is I am putting together something which will be programmed by others, and they may want it.

[TheCalligrapher]

EDIT: I found I do have vsnprintf but not vsprintf. I know that in embedded work one should never use the latter. So maybe this is a better way to go to get vsprintf.

What is so special about embedded to claim that "one should never use the latter" in "embedded work" specifically?

If you hold such a radical stance towards `vsprintf`, it should probably apply to any kind of work, shouldn't it?

[peter-h]

I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?

[IanB]

I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?

I don't follow your comments. vsnprintf is identical to vsprintf, except vsnprintf is safer. If you need vsprintf, simply use vsnprintf instead.

On the other hand, vsprintf and sprintf have different argument lists. They are not replaceable equivalents. But neither of them should be used when vsnprintf and snprintf exist.

[peter-h]

I was just after help on my question

I also didn't want to get into complications, but the explanation is in the 1st link in my 1st post. I currently have vsprintf but it isn't thread-safe because it lives in the ST-supplied libc.a standard C library which uses (probably) statics and (certainly) the heap. And whether you like it or not, people do use sprintf and vsprintf, and if they do they will get all sorts of funny problems. My sprintf is however thread-safe because I have a nice library for most of these, but it doesn't have vsprintf.

So I have two options: create a vsprintf from one of the other (thread-safe) functions, or undefine it so people can't use it. I tried removing it from the ST lib (objcopy strip symbol option) but that doesn't work because that lib (compiled, no source) has vsprintf defined (it has the code for it) and objcopy doesn't allow the symbol removal in such a case.

Is it hard to get a vsprintf from sprintf, with a function, or a macro?

For example, I have the following from that open source printf.c mentioned earlier. A "v" function calls va_start and va_end around the printf.

Code: [Select]

int printf_(const char* format, ...)
{
  va_list va;
  va_start(va, format);
  char buffer[1];
  const int ret = _vsnprintf(_out_char, buffer, (size_t)-1, format, va);
  va_end(va);
  return ret;
}


int sprintf_(char* buffer, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const int ret = _vsnprintf(_out_buffer, buffer, (size_t)-1, format, va);
  va_end(va);
  return ret;
}


int snprintf_(char* buffer, size_t count, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const int ret = _vsnprintf(_out_buffer, buffer, count, format, va);
  va_end(va);
  return ret;
}


int vprintf_(const char* format, va_list va)
{
  char buffer[1];
  return _vsnprintf(_out_char, buffer, (size_t)-1, format, va);
}


int vsnprintf_(char* buffer, size_t count, const char* format, va_list va)
{
  return _vsnprintf(_out_buffer, buffer, count, format, va);
}


int fctprintf(void (*out)(char character, void* arg), void* arg, const char* format, ...)
{
  va_list va;
  va_start(va, format);
  const out_fct_wrap_type out_fct_wrap = { out, arg };
  const int ret = _vsnprintf(_out_fct, (char*)(uintptr_t)&out_fct_wrap, (size_t)-1, format, va);
  va_end(va);
  return ret;
}


[langwadt]

I would think this can be done with a macro but googling didn't come up with one.

vsprintf from vsnprintf. You have to work out the value for the parameter; not obvious.

But vsprintf from sprintf is just a substitution of the parameters, no?

I use https://github.com/eyalroz/printf  it implements vsprintf as:

Code: [Select]

int vsprintf_(char* s, const char* format, va_list arg)
{
  return vsnprintf_(s, PRINTF_MAX_POSSIBLE_BUFFER_SIZE, format, arg);
}


[IanB]

I was just after help on my question

I also didn't want to get into complications, but the explanation is in the 1st link in my 1st post. I currently have vsprintf but it isn't thread-safe because it lives in the ST-supplied libc.a standard C library which uses (probably) statics and (certainly) the heap. And whether you like it or not, people do use sprintf and vsprintf, and if they do they will get all sorts of funny problems. My sprintf is however thread-safe because I have a nice library for most of these, but it doesn't have vsprintf.

So I have two options: create a vsprintf from one of the other (thread-safe) functions, or undefine it so people can't use it. I tried removing it from the ST lib (objcopy strip symbol option) but that doesn't work because that lib (compiled, no source) has vsprintf defined (it has the code for it) and objcopy doesn't allow the symbol removal in such a case.

Is it hard to get a vsprintf from sprintf, with a function, or a macro?

For example, I have the following from that open source printf.c mentioned earlier. A "v" function calls va_start and va_end around the printf.

Code: [Select]

...

I'm just really not able to understand what you are getting hung up on.

In your first post you said you have vsnprintf available.

In the code you listed above, vsnprintf is there.

Why can you not simply use vsnprintf?

For example, vsprintf looks like this:

Code: [Select]

int vsprintf (char * s, const char * format, va_list arg );
While vsnprintf is like this:

Code: [Select]

int vsnprintf (char * s, size_t n, const char * format, va_list arg );
The only difference is the size of the output buffer, n. If you don't know the size of the buffer, just put in a big arbitrary number for n, like 10000 or something. If you complain that this is unreasonable and it may cause a buffer overrun, well this is exactly why you shouldn't be using the unsafe version in the first place.

Also, if you don't know the size of the buffer, the code is already in big trouble.

[peter-h]

langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

[SiliconWizard]

langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

Of course. If you pass a very large value to it, then it essentially becomes equivalent.

I would still personally deprecate the use of vsprintf() in your system and warn your users about it. They won't hold your willingness to make the code more secure against you. As long as you help them a little:

One way of doing that without angering them to no end is to use a macro that, when defined, will allow the use of deprecated functions. (Alternatively, you can invert the semantics in order for your users not to have to take extra steps, and make it so that when a specific macro is defined, it will prevent the code from compiling if using some of the deprecated functions. Then explain in your docs in a paragraph the rationale and how to handle it.)

[langwadt]

langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

might want to do:

Code: [Select]


__attribute__((deprecated("Use vsnprintf instead"))) int vsprintf(char* s, const char* format, va_list arg)
{
  return vsnprintf(s, PRINTF_MAX_POSSIBLE_BUFFER_SIZE, format, arg);
}


[peter-h]

Thanks - that's an interesting way to do a compile-time warning.

Re sprintf, I have some code which is full of these (ex ST, in fact ex FreeRTOS actually! - it is used to generate their dynamic task listing) and what they are doing is emitting fairly short strings of say 20 bytes but a lot of them, into a buffer whose size has been experimentally determined as needing ~1200 bytes, so I set it at 1500.

One can get the #bytes each snprintf has output (or tried to output, in case of truncation) but people tend to be too lazy to be adding up these bits as they go along.

And then if the buffer is on the stack (by far the easiest way, because it gets chucked out upon exit) its size is fixed anyway.

One could code it but as I say people tend to be lazy.

One couldd also prefill the buffer with something and before each sprintf check that value is still there at buffer[sizeof(buffer)-20], say.

[PlainName]

One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.

[SiliconWizard]

One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.

Yeah. As I've understood, peter-h designs products the customers of which can customize the firmware, so that would explain his position.

While you can choose who you work with (if you're the boss at least), it's a lot harder to choose your customers. Or that might mean giving up on a number of them, or spending endless hours on support. So I feel the pain. (Peter can correct me if I didn't get his constraints properly though...)

[tellurium]

Another possibility would be to add some small scripting engine to the product, import a bunch of performance-critical functions from C, and provide customers with a scripting API instead of a C API.

My company did that a lot, we had customers that built their products entirely on a scripting engine.

[Berni]

langwadt - thanks; that is perfect. I didn't realise one can use the "n" version and use a very large buffer value.

Of course. If you pass a very large value to it, then it essentially becomes equivalent.

I would still personally deprecate the use of vsprintf() in your system and warn your users about it. They won't hold your willingness to make the code more secure against you. As long as you help them a little:

One way of doing that without angering them to no end is to use a macro that, when defined, will allow the use of deprecated functions. (Alternatively, you can invert the semantics in order for your users not to have to take extra steps, and make it so that when a specific macro is defined, it will prevent the code from compiling if using some of the deprecated functions. Then explain in your docs in a paragraph the rationale and how to handle it.)

Yep this macro is a good solution in my opinion too.

If they try to use the unsafe versions they will get a long descriptive error of why this is bad. But at the same time they have the option of adding a macro define that turns it into a regular warning and lets the code compile anyway by passing the unsafe versions over to the safe versions by telling them they have an effectively infinite buffer size.

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output. It becomes a problem when you start using %s that could potentially make the string very long, especially if the data is coming from an outside source. That's where the safe versions should absolutely be used. Those things are the source of most security exploits.

[IanB]

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output.

On the other hand, if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.

[Berni]

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output.

On the other hand, if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.

Yes but that means you are either using %s to join strings, or you are moving your pointer down along the buffer. In that case you no longer have a guarantee of having enough space.

But in general on MCUs you don't tend to build very long strings. It is common practice to just build the part of the string you need and imidietly flush it out to UART, on a display, onto a SD card..etc since you only have a few kilobytes of RAM to work with anyway.

[PlainName]

One could code it but as I say people tend to be lazy.

Not sure I would want to be associated with any product programmed by that sort of developer.

Yeah. As I've understood, peter-h designs products the customers of which can customize the firmware, so that would explain his position.

Fair enough. I'd still want to offload technical support, though

[SiliconWizard]

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break.

Sure, but it still relies on making assumptions instead of having a safeguard. I have surely used sprintf() a lot in the past, as many of us, I'm sure. I don't anymore.

It doesn't cost you any more work to use snprintf(), a fixed-size buffer and passing the size of said buffer to snprintf. The runtime overhead is negligible. In the possibly rare event of the string being truncated because you didn't choose an appropriate buffer size, or because there was an unexpected input, then it will just truncate the resulting string instead of crashing. You can still check the return value to see if it fits or not.

This is part of a more general approach of doing (almost systematic) "input validation", which I'm a strong proponent of.

[nctnico]

That being said unsafe sprintf is not always bad. If you are feeding in well known parameters and using a large buffer with some headroom it won't ever break.

Sure, but it still relies on making assumptions instead of having a safeguard. I have surely used sprintf() a lot in the past, as many of us, I'm sure. I don't anymore.

It doesn't cost you any more work to use snprintf(), a fixed-size buffer and passing the size of said buffer to snprintf. The runtime overhead is negligible. In the possibly rare event of the string being truncated because you didn't choose an appropriate buffer size, or because there was an unexpected input, then it will just truncate the resulting string instead of crashing. You can still check the return value to see if it fits or not.

This is part of a more general approach of doing (almost systematic) "input validation", which I'm a strong proponent of.

I agree. Especially if customers (read: noobs) are going to write code, only make snprintf() available. On embedded systems the buffer is more likely to be too small rather than too large causing all kinds of weird behaviours / hard to catch bugs due to overwriting other data. In the end it won't help confidence in the product might raise all kinds of support questions.

[peter-h]

Not sure I would want to be associated with any product programmed by that sort of developer.

Thank you for the compliment but you don't have to be In this case the customer will write his own module and it is unlikely you will end up using it.

Peter can correct me if I didn't get his constraints properly though...)

In this case that's correct, but actually a vast amount of ST supplied code uses just an sprintf... Looking around, snprintf is rarely used.

we had customers that built their products entirely on a scripting engine.

A lot of work and a lot of RAM. Also a lot of people know C well enough.

using a large buffer with some headroom it won't ever break. For example doing "Values: %d, %d, %d" can only create so large of an output. It becomes a problem when you start using %s that could potentially make the string very long, especially if the data is coming from an outside source. That's where the safe versions should absolutely be used. Those things are the source of most security exploits.

Exactly.

if you are doing many successive small writes to gradually fill a large buffer, you could still run off the end of the buffer if you miscalculate the number of writes or the allocated buffer size.

Recently I dealt with such a case by planting a marker 100 bytes before the end of a 1500 byte buffer, and checking it before each "tiny sprintf". That code produced a long table



If the marker is gone, you get a ** BUFFER OVERFLOW ** in there. Most of the values come from deep down in somebody else's (FreeRTOS actually) code.

But in general on MCUs you don't tend to build very long strings. It is common practice to just build the part of the string you need and imidietly flush it out to UART, on a display, onto a SD card..etc since you only have a few kilobytes of RAM to work with anyway.

Exactly.