Author Topic: [Solved] Saving "printf" arguments for later? Real-time on slow processor.  (Read 7971 times)

0 Members and 2 Guests are viewing this topic.

Online magic

  • Super Contributor
  • ***
  • Posts: 7529
  • Country: pl
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #100 on: January 21, 2025, 10:48:27 am »
We use it because it's convenient, but it's not efficient for sure. The fact that printf is based on analyzing format and parameters at run time kind of bites. The only objective benefit of doing this at run time would be to use formats defined at run time, and while that's something you're allowed to do, it's usually not recommended - pretty dangerous. So with fixed (at compile time) formats, this is merely because of language limitations.
It's space efficient, though, because "%d" is just two bytes which tends to be less than the assembly it would take to call a function (including a pointer to the function).

So printf is actually quite sensible in MCU applications, or for logging in situations where the CPU overhead is acceptable. (Besides, I'm not sure if scanning the format string significantly adds to the cost of actually printing those numbers, particularly on MCUs lacking hardware MUL/DIV instructions).

I became a fan of printf after seeing what happens to my flash usage after switching to custom print_string, print_int and print_float functions. The functions themselves were smaller than a printf implementations (no surprise, they didn't even have all its functionality) but the flash image became larger overall.

Which brings as back to everyone's favorite topic: R&$# sucks as much as C++ and both are inferior to C :D (at least when it comes to systems and embedded programming).
« Last Edit: January 21, 2025, 10:58:33 am by magic »
 
The following users thanked this post: mikerj, Siwastaja, Nominal Animal

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21453
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #101 on: January 21, 2025, 11:07:22 am »
Yeah, with any important project, analysis of input ranges and how computational accuracy affects the output is a must. Usually it's not even rocket science so I suggest trying to do it even in non-critical projects. As a side effect of knowing how to analyze your ranges, efficient fixed point number formats become available. (Or turning this around: with fixed point representations you need to think about what is the largest number that needs to be presented, for every step, given worst-case input parameters; which is extra work. But it's really not a bad idea to do that anyway, in which case fixed point is not any more "extra work" after all.)

The rationale for floating point really is convenience; let the machine handle automatically finding the balance between resolution of the smallest step vs. capability of store the largest possible number so that the programmer "doesn't need to think about accuracy and ranges at all". The downside is either a more complex hardware, or slow software implementation, and some bits of "overhead" (like, if you know your ranges and can fix them without extra safety margin, 24-bit integer gives you similar resolution to a 32-bit float).

Another downside of floating point is that in some rare cases programmer should have been thinking about accuracy and ranges after all. Especially the single-precision float gives false sense of security; always use doubles (which is clearly why e.g. C language tends to do that by default) when you don't want to think.

Unfortunately pre-analysis is "difficult" in any large application or one where the the inputs cannot be well constrained.

An example of the latter is anything involving matrix inversion, e.g. a Spice analysing a circuit. It is far from unknown for apparently trivial changes (e.g. node names or order) to the schematic/netlist to cause simulation to succeed or fail.

It is also amusing to see how how well C coped with floating point computations where the hardware used an 80-bit internal representation, not 64-bit. Not sure whether that still happens; moved away from that mess long ago, thankfully.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21453
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #102 on: January 21, 2025, 11:09:54 am »
We use it because it's convenient, but it's not efficient for sure. The fact that printf is based on analyzing format and parameters at run time kind of bites. The only objective benefit of doing this at run time would be to use formats defined at run time, and while that's something you're allowed to do, it's usually not recommended - pretty dangerous. So with fixed (at compile time) formats, this is merely because of language limitations.

I became a fan of printf after seeing what happens to my flash usage after switching to custom print_string, print_int and print_float functions. The functions themselves were smaller than a printf implementations (no surprise, they didn't even have all its functionality) but the flash image became larger overall.

Presumably you did omit the printf image from the a.out file.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Online magic

  • Super Contributor
  • ***
  • Posts: 7529
  • Country: pl
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #103 on: January 21, 2025, 11:45:57 am »
I inspected the binaries with objdump and there was no printf implementation in them, but the size of logging functions blew up. IIRC part of it was also register moves, because if you calculate a bunch of values which will be passed to printf later you can place them in the right registers in advance, but if you call print_int repeatedly, all values must be in the same "first" register, so there is no way to avoid moves between calls.

Another factor is that if you call printf("x=%d y=%d z=%d\n") you only call printf once, but if you replace it with a sequence of low-level calls, you end up with four calls to print_string. So further overhead is: three call instructions, three pointers to the print_string function embedded in those instructions, three string pointers, three instructions (or instruction sequences) to load them into some register.

The whole thing happened because the project was at the limits of available flash and I was looking for the smallest printf implementation or something better than printf. Ended up with some off the shelf minimized printf. And it wasn't even printing that much, I think it had a two digit number of % signs overall, so I was quite surprised to see it working in printf's favor. And yes, I was comparing against stripped down printf implementations, but with hundreds of % signs even a fully featured printf would win.
« Last Edit: January 21, 2025, 12:26:01 pm by magic »
 
The following users thanked this post: tggzzz, Siwastaja

Offline AVI-crak

  • Regular Contributor
  • *
  • Posts: 141
  • Country: ru
    • Rtos
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #104 on: January 21, 2025, 05:02:22 pm »
I don't understand the way to solve your problem, I can't make sense of the conversation. It doesn't make sense.
You have a huge log in front of you, and everyone is rolling it around a huge field. The log won't get smaller, because it's a log, it has to be big.
Any variant of the classic printf() is a log. As long as there is a format string - the log will not disappear.
You need speed and code size - then you should give up the format string.

Translated with DeepL.com (free version)
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 4061
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #105 on: January 21, 2025, 06:03:22 pm »
It is also amusing to see how how well C coped with floating point computations where the hardware used an 80-bit internal representation, not 64-bit. Not sure whether that still happens; moved away from that mess long ago, thankfully.

No.  The x87 is essentially deprecated and SSE doesn't support 80 bit floats, nor does any current non-intel platform.  Most people decided they would rather have (closer to) deterministic output rather than the best precision.  The way that most C compilers would handle the x87 was that temporary variables would be 80 bit as long as they were stored in registers, but would be truncated to 64 bit when spilled to memory.  This made the extra precision dependent on exactly how the code was compiled.

You can still use long double on most x86_64 platforms to get 80 bit floats that will use the x87.  In that case the memory and register representation will be 80 bits, but you pay a performance hit in most applications due to the more expensive load/store operations.

As partial compensation, we now have FMA operations that can reduce truncation errors in multiply accumulate operations.
 
The following users thanked this post: oPossum

Offline cv007

  • Frequent Contributor
  • **
  • Posts: 864
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #106 on: January 21, 2025, 08:24:30 pm »
Quote
I already added buffering to _write
I searched the thread to see if some other info provided is contrary to this (didn't see anything), but this implies you are using printf and not sprintf. If you are going through printf to get your _write to output to a buffer, maybe you should instead be going to the buffer directly via snprintf. The speed difference between printf and snprintf could be quite a bit depending on what stdio libraries are in use (we still don't know I guess, so just assume newlib nano).

Code: [Select]
//replacement for printf, redirect to vsnprintf
int printf( const char* fmt, ... ){
    va_list ap;
    va_start( ap, fmt );
    //buf and siz according to remaining buffer space
    int rv = vsnprintf( buf, siz, fmt, ap );
    //adjust buffer space used/remaining
    va_end( ap );
    return rv;
}

 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #107 on: January 21, 2025, 08:53:26 pm »
Quote
I already added buffering to _write
I searched the thread to see if some other info provided is contrary to this (didn't see anything), but this implies you are using printf and not sprintf. If you are going through printf to get your _write to output to a buffer, maybe you should instead be going to the buffer directly via snprintf. The speed difference between printf and snprintf could be quite a bit depending on what stdio libraries are in use (we still don't know I guess, so just assume newlib nano).

Code: [Select]
//replacement for printf, redirect to vsnprintf
int printf( const char* fmt, ... ){
    va_list ap;
    va_start( ap, fmt );
    //buf and siz according to remaining buffer space
    int rv = vsnprintf( buf, siz, fmt, ap );
    //adjust buffer space used/remaining
    va_end( ap );
    return rv;
}

Sorry, I should have been more clear. I went through a systemic process of reworking printing in the past leading up to this post. I used to use _write in the past. I'm currently using snprintf for writing to thread local buffers.

I ended up doing a benchmark of the snprintf wrapper that I supply to the library. I give it 8 large 32 bit numbers (1 hex, and 7 decimal). I found it takes 10000 cycles round trip to put the formatted string into the buffer. The library can easily do 10 of these in a row within the "critical section", which would take about 10ms, it is not surprising it was failing to reliably hit a ~1ms window. I would bet different numbers could result in substantially different speeds.

I determined that I actually need to get through the function call in less than 500 cycles - which I think I'll be able to do by saving the format string and all the arguments to a FIFO for later processing. (roughly 64 bytes of format string, and 32 bytes of numbers)
« Last Edit: January 21, 2025, 09:13:56 pm by incf »
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #108 on: January 21, 2025, 09:31:33 pm »
One question: have you figured out exactly which number formats are actually used by your logging code, and which ones you can ignore?

I ask because there's been a lot of stuff posted here about floats and whatnot. If it turns out that you're not logging any floating-point #s then we can just ignore that format entirely.
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #109 on: January 21, 2025, 10:16:45 pm »
One question: have you figured out exactly which number formats are actually used by your logging code, and which ones you can ignore?

I ask because there's been a lot of stuff posted here about floats and whatnot. If it turns out that you're not logging any floating-point #s then we can just ignore that format entirely.

It's all large 32 bit integers and strings. Often 8 per snprintf call.

And in 500 CPU cycles, there is not a lot I can do except copy the request into a buffer for later.

The side threads around optimization are very interesting, but not directly applicable for this particular issue.
« Last Edit: January 21, 2025, 10:18:43 pm by incf »
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #110 on: January 21, 2025, 10:46:13 pm »
So another question, hopefully one that won't further muddy the waters:

Your scheme seems sound: on receipt of a logging request, copy the format string and the values to be logged to a buffer. Fine and dandy.

Then when you have a spare moment, retrieve that stuff from the buffer and display it. Also fine and dandy.

But what happens if your displaying (or saving to disk or whatever) code gets interrupted by a high-priority request to do whatever your thingy is doing (monitoring the network, etc.)? You'd have to drop your logging handling code, correct?

Is it OK if you lose some of the logged data, or is every bit of it precious?

Or could you simply suspend the logging thread, then resume it after the high-priority code has completed? You'd have to have coordination between your main processing thread(s) and the logging thread anyway, so you can remove what you output from the buffer.

I'm sure you've already though of some or all of this. But those of us out here are curious ...
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 4061
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #111 on: January 21, 2025, 10:48:26 pm »
Quote
It's all large 32 bit integers and strings. Often 8 per snprintf call.

Strings as in literal parts of the format string or %s format specifiers?
« Last Edit: January 21, 2025, 10:53:18 pm by ejeffrey »
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #112 on: January 21, 2025, 10:51:44 pm »
If he's logging strings (%s specifiers), then he'd need to save both the format string ("User: %s") and the string that gets inserted into the format, probably in a memory heap.

That would complicate the buffering scheme, but not unduly.
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #113 on: January 21, 2025, 10:56:05 pm »
So another question, hopefully one that won't further muddy the waters:

Your scheme seems sound: on receipt of a logging request, copy the format string and the values to be logged to a buffer. Fine and dandy.

Then when you have a spare moment, retrieve that stuff from the buffer and display it. Also fine and dandy.

But what happens if your displaying (or saving to disk or whatever) code gets interrupted by a high-priority request to do whatever your thingy is doing (monitoring the network, etc.)? You'd have to drop your logging handling code, correct?

Is it OK if you lose some of the logged data, or is every bit of it precious?

Or could you simply suspend the logging thread, then resume it after the high-priority code has completed? You'd have to have coordination between your main processing thread(s) and the logging thread anyway, so you can remove what you output from the buffer.

I'm sure you've already though of some or all of this. But those of us out here are curious ...

The OS and the IPC (interprocess call) scheme handles all of that. Race conditions, data contention, etc. are all completely impossible by design. If I tried to do "something funny" it simply would not compile.

The data is not precious. It would be truncated if a FIFO got full.

There is a lot of idle time, and the buffers are huge. I consider it practically impossible to fill the FIFO enough to lose data.

The device is only awake and actively doing real-time work seemingly less than a 0.1% of the time. The remainder is free for the processor to clean up the mess made by the real-time process before powering down.
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #114 on: January 21, 2025, 11:00:31 pm »
Quote
It's all large 32 bit integers and strings. Often 8 per snprintf call.

Strings as in literal parts of the format string or %s format specifiers?
If he's logging strings (%s specifiers), then he'd need to save both the format string ("User: %s") and the string that gets inserted into the format, probably in a memory heap.

That would complicate the buffering scheme, but not unduly.

Yes, there are %s strings that I have to copy into the buffer. It will make things more complicated but is fine.

No heap, we will probably statically allocate some data structure specifically for this. Some sort of ring buffer filled with variable length/layout structures.

The heap scares me, it seems so nondeterministic (at least without looking at how exactly libc decides to implement it). Although it would be so much easier to write if I was using a heap.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15933
  • Country: fr
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #115 on: January 21, 2025, 11:01:03 pm »
Just a question, but is this amount of logging really useful? Can you not ignore at least part of it?
 

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #116 on: January 21, 2025, 11:08:15 pm »
Just a question, but is this amount of logging really useful? Can you not ignore at least part of it?
Unfortunately, the majority of it is relevant some portion of the time - or rather, the context it provides is even if the data sometimes is redundant. The most valuable logging data comes directly from private/static variables the "real-time" sections. Not to mention the limitations around printing being "all on" or "all off."

I don't really want to build hash tables when a fast buffering scheme will do it (there are too many seemingly identical printf statements in radically different contexts to feasibly sort out).

It's a state machine whose internal state is not visible except through human readable logs (and we don't want to get into the business of string parsing since it's not going to solve out real problem the formatted strings have to be computed first before they can be parsed, and many lines have to be parsed together as a unit for anything useful to be learned, and all of that happens 10-20 milliseconds after we've missed out real-time deadline)
« Last Edit: January 21, 2025, 11:19:14 pm by incf »
 

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21453
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #117 on: January 21, 2025, 11:14:25 pm »
Yes, there are %s strings that I have to copy into the buffer. It will make things more complicated but is fine.

No heap, we will probably statically allocate some data structure specifically for this. Some sort of ring buffer filled with variable length/layout structures.

The heap scares me, it seems so nondeterministic (at least without looking at how exactly libc decides to implement it). Although it would be so much easier to write if I was using a heap.

Are the strings immutable, i.e. stored in ROM? If so, simply pass their address. Immutability is a powerful property in multithreaded environments. Alternatively pass the string the first time it is used in a printf, thereafter pass a pointer or index.

You are right to fear heaps, except for stuff malloc()ed during start-up and never free()d. In general it is much better to rely on a decent modern GC. That is likely to be correct and faster than something thrown together by someone who vaguely remembers refcounts from university lectures based on historic concepts. Irrelevant in your case, of course.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Online tggzzz

  • Super Contributor
  • ***
  • Posts: 21453
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #118 on: January 21, 2025, 11:19:57 pm »
Not to mention the limitations around it being "all on" or "all off."

Ah, you need a "come from" mechanism so your printf can determine the component doing the logging, and hence whether to log a particular message.

Easy in a language with reflection, but you might be able to peek at the stack to see the return address.  >:D
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #119 on: January 21, 2025, 11:20:17 pm »
No heap, we will probably statically allocate some data structure specifically for this. Some sort of ring buffer filled with variable length/layout structures.

The heap scares me, it seems so nondeterministic (at least without looking at how exactly libc decides to implement it). Although it would be so much easier to write if I was using a heap.

I get that; heaps are kinda scary. I've written heap-management routines that worked perfectly well, but it always took a lot of head-scratching and drawing stuff on paper.

If you've got a gazillion bytes of memory then it'll be a lot easier just to allocate fixed-length elements (you do know what the longest strings you'll be dealing with are, right?). Easy-peasy then.
 
The following users thanked this post: 5U4GB

Offline incfTopic starter

  • Regular Contributor
  • *
  • Posts: 109
  • Country: us
  • ASCII > UTF8
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #120 on: January 21, 2025, 11:28:39 pm »
No heap, we will probably statically allocate some data structure specifically for this. Some sort of ring buffer filled with variable length/layout structures.

The heap scares me, it seems so nondeterministic (at least without looking at how exactly libc decides to implement it). Although it would be so much easier to write if I was using a heap.

I get that; heaps are kinda scary. I've written heap-management routines that worked perfectly well, but it always took a lot of head-scratching and drawing stuff on paper.

If you've got a gazillion bytes of memory then it'll be a lot easier just to allocate fixed-length elements (you do know what the longest strings you'll be dealing with are, right?). Easy-peasy then.

I've got 500 cycles to work with. I'm going with variable sized elements for "reasons".  (namely, a large swaths of prints occasionally seem to be little more than a single %d or %u done in loops to print arrays. Some prints are long strings with 10 fields, other are not)

Allocating 10 chunks of memory* from the heap and copying/populating the data/fields in only 500 clock cycles seems like it would be hard to do without a lot of consideration.

*1x 64 bytes, 1x48 byte, and 8x 16 bytes (or similar, sometimes there will be strings)
« Last Edit: January 21, 2025, 11:37:22 pm by incf »
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #121 on: January 21, 2025, 11:32:20 pm »
I'm going with variable sized elements for "reasons".  (namely, a large swaths of prints occasionally seem to be little more than a single %d or %u done in loops to print arrays)

So I take it your variable-sized elements include a size prefix (like those non-ASCIIZ C strings), yes?
If you used fixed-length allocations you'd have a lot less bookkeeping to do.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 7308
  • Country: fi
    • My home page and email address
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #122 on: January 22, 2025, 12:12:41 am »
A temporary sidetrack:

what is the accuracy of X+Y, X-Y, X*Y, X/Y ?
There is an easy but very informative and useful programming exercise here.

Implement X and Y as random numbers, with some distribution around some median.  Error bounds are by convention those that encompass 68.3% of all values symmetrically around the median, 34.15% above the median, and 34.15% below the median.  Repeat one of the operations above, creating a histogram of the results.  After a sufficient number of iterations, find the median, and the error bounds.  (For a normal distribution, the error bounds are exactly one standard deviation below and above the median.)

The same approach can be applied to investigate "black box" multivariate functions numerically, that for some reason or another cannot be solved analytically.  Each result distribution per fixed input argument the represents one data point.  The function can then be characterized in the desired input argument domain to any desired fidelity, by having enough data points.

To generate random numbers in any distribution \$P(x)\$, first calculate or find out the cumulative distribution function \$F_x(x)\$,
$$F_x(x) = \int_{-\inf}^x P(\chi) \, d\chi$$
noting that \$F_x(-\inf) = 0\$ and \$F_x(+\inf) = 1\$.  To generate a random number \$x\$, you first generate an uniform random number \$p\$, \$0 \le p \le 1\$, and then find \$x\$ such that \$F_x(x) = p\$.  You can either calculate or approximate the inverse function, or you can use a binary search, because \$F_x(x)\$ is a nondecreasing function in \$x\$.

For a normal distribution with median/average \$\mu\$ and standard deviation \$\sigma\$,
$$F_x(x) = \frac{1}{2} + \frac{1}{2} \operatorname{erf}\left( \frac{x - \mu}{\sqrt{2} \sigma} \right)$$
where \$\operatorname{erf}()\$ is the error function.  Often the Box–Muller method is preferable, though.

A variant is where you model the distribution as a simple interval, as a rectangular distribution.  Within this interval, each value is equally likely; outside the range the probability is zero.  This is called interval arithmetic.  Simply put, each variable Z actually refers to a continuous interval, (Zmin..Zmax).  Arithmetic binary operators are applied all four ways, with the result consisting of the minimum and the maximum of those results.

Because this distribution is rather unrealistic, it gives different results than when one uses the standard distribution (Gaussian) or any other more realistic distribution, but it can nevertheless be useful in some cases.  For example, given X = 200±1% = 198..202 and Y = 100±1% = 99..101, interval arithmetic yields
  • X + Y = 297 .. 303 = 300±1%
  • X - Y = 97 .. 103 = 100±3%
  • X * Y = 19602 .. 20402 ≃ 20002±2%
  • X / Y ≃ 1.96 .. 2.04 = 2±2%

I like to point out these kind of programming experiments, because it is how I personally gain a more intuitive understanding of how imprecise values behave.  One does not need to be a mathematician to be able to understand them and to use them to solve problems.
« Last Edit: January 22, 2025, 12:15:18 am by Nominal Animal »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 7308
  • Country: fi
    • My home page and email address
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #123 on: January 22, 2025, 12:27:17 am »
It's all large 32 bit integers and strings. Often 8 per snprintf call.

And in 500 CPU cycles, there is not a lot I can do except copy the request into a buffer for later.
Tight.

One option is to create a hash table of formats you have already parsed, in RAM at run time, with the target specifying what parameters you need to copy to the circular buffer.  If the format strings are all in Flash, they don't need to be copied; otherwise, the string itself has to be copied to the circular buffer.  If a %s points to Flash, only the pointer needs to be copied; otherwise you'll need to copy the contents also.  Essentially, you'll need to copy (from the variadic argument list) native words including pointers, native doublewords (if 64-bit values are supported), and RAM-based strings (because those are likely overwritten after the call).  If you use a single 32-bit number, two bits per item suffices, with 00 indicating end, allowing you to support up to 16-parameter formats.  This means that the circular buffer entries should have variable length, so start with the length of each record.

If the format is not in the hash table, you parse it, adding the data to the circular buffer, plus constructing the 32-bit number.  If you had fewer than 17 parameters to format, then you add the mapping to the hash table.  If there are just one or two parameters formatted, it might be faster to just re-parse it each time, and leave the limited hash table for the larger format strings.

This does mean that you end up parsing the format strings twice, but in 500 cycles, I think it is a compromise you'll have to do.
« Last Edit: January 22, 2025, 12:29:06 am by Nominal Animal »
 

Offline Analog Kid

  • Super Contributor
  • ***
  • Posts: 1315
  • Country: us
Re: Saving "printf" arguments for later? Real-time on slow processor.
« Reply #124 on: January 22, 2025, 01:19:44 am »
Essentially, you'll need to copy (from the variadic argument list) native words including pointers, native doublewords (if 64-bit values are supported), and RAM-based strings (because those are likely overwritten after the call).

You might want to read the posts here more carefully. The OP has already written
Quote
It's all large 32 bit integers and strings. Often 8 per snprintf call.

No 64-bit values.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf