Disabling Compiler Optimization vs Using volatile

[John Coloccia]

Across time and space the general advice has always been to compile without optimizations unless you need them.

I realized what John Coloccia's misunderstanding here probably is; indeed I have never heard this "general advice" anywhere, but instead, I have heard many times that "premature optimization is the root of evil", and similar remarks. Which is generally useful idea; correct to an extent. Maybe John Coloccia misunderstood this advice and thought it means compiler optimization settings, but that's really not the point. We are advised against premature manual (micro-)optimization because it easily leads to harder-to-understand, harder-to-maintain code, and also because knowing what the actual effects are is surprisingly difficult.

Instead, really the general advice is to try to write as high-level code as possible, designed for understanding, maintainability, correctness and error handling, and then let the compiler figure out how to micro-optimize it.

Don't take that thinking too far, though - this principle only applies to manual micro-optimization, and micro means small localized adjustments. Choosing efficient data structures and algorithms is sometimes of paramount importance and compilers can't do anything about it if you chose a O(n^2) sorting algorithm which takes 100x too long, or chose a memory representation of data which takes 2x the available RAM.

Also worth noting that if one enjoys manual micro-optimizations and gets them right, who are we to say it should not be done. But compiling at -O0 and then manually micro-optimizing instead would be probably worst use of one's time ever.

Well, yeah, you should definitely not pre-optimize what you're writing as high-level code for sure. If you really need the speed, the compiler will almost certainly optimize the code better than you can. And trying to get clever at the high-level can often result in WORSE generated code. Compiler designers have gotten quite good at figuring out what you intend to do, and hand optimization often just confuses things.

It's more about understanding the software you're writing, understanding where the pain points are, if any, and keeping things as simple as possible. If you can get away with no optimizations, there's absolutely no reason to do anything else. Also, as optimizations go, the vast majority of the time that I've needed something optimized it was because the ALGORITHM sucked. Poor memory organization, misunderstanding (or lack of understanding) how caches work, pipelining etc. All the compiler optimizations in the world aren't going to help with that, and that's most often where I see it go badly and where I get the huge gains, if I need them. The compiler optimization is a band-aid in those cases, but the real problem is a terrible algorithm.

Here's a great example....sitting inside an outer loop:

Code: [Select]

outer loop {
    //do a bunch of stuff...

     /*
      * This looks funny. Please leave it alone unless
      * you know what you're doing and can make it faster.
      * This is optimized for the processor.
      */
        if (data[tmpy * FRAME_WIDTH + tmpx] > 253) {
                if (data[tmpy * FRAME_WIDTH + tmpx] == 255) {
                    //do a bunch more stuff
                }
                break;
        }
}

Why not just check for ==255? I will tell you that it runs VERY slowly on this particular processor if you do that. I know more about the problem domain than the compiler does, and it's way more important to keep the instruction pipeline from flushing than it is to save a few instructions. With optimizations ON, in fact, it's practically impossible to arrange to get the code I want to make the processor reliably do what I want.

So at the end of the day, I recommend you understand your pain points before doing any optimization at all, and see what it does. If you then find that you want to optimize for size and/or speed because it's useful for your particular project, go for it. But if it's not necessary, then it can ONLY make things worse. If I have a tight, real-time requirement, and I'm hitting it with margin, hitting it with more margin is just not normally helpful.

I'm more or less constantly telling people "stop being clever...just go simple, and see if it works." And they feel like it's wasteful...and I'm like, "Would you rather it just sit around twiddling it's thumbs for the next 500us?"

[brucehoult]

Why not just check for ==255? I will tell you that it runs VERY slowly on this particular processor if you do that.

Weird.

I know more about the problem domain than the compiler does, and it's way more important to keep the instruction pipeline from flushing than it is to save a few instructions.

But surely slow comparison to 255 is EXACTLY what should be in the compiler's knowledge? It's certainly not problem domain.

And why not?

Code: [Select]

uint8_t val = data[tmpy * FRAME_WIDTH + tmpx];
if (val > 253 && val == 255) {
  ...
}

Mind you, any decent compiler is going to delete the 253 test anyway.

WHY is the 255 test slow?

[John Coloccia]

Why not just check for ==255? I will tell you that it runs VERY slowly on this particular processor if you do that.

Weird.

I know more about the problem domain than the compiler does, and it's way more important to keep the instruction pipeline from flushing than it is to save a few instructions.

But surely slow comparison to 255 is EXACTLY what should be in the compiler's knowledge? It's certainly not problem domain.

And why not?

Code: [Select]

uint8_t val = data[tmpy * FRAME_WIDTH + tmpx];
if (val > 253 && val == 255) {
  ...
}

Mind you, any decent compiler is going to delete the 253 test anyway.

WHY is the 255 test slow?

The code itself isn't slow, but the predictive branching in this case always predicts wrong and forces a pipeline flush if you code it sanely. This is sort of an extreme case, but mostly the optimizations I've found that make the biggest differences are related to memory, pipelines and things like that.

As an example, I have many other cases where I've ripped out very code and memory efficient algorithms, and replaced them with brute force searches through a large array. Extremely inefficient and "wasteful" on the surface, but in practice it absolutely screams because I can keep my instruction pipeline from breaking, eliminate instruction cache misses, and minimize memory cache misses. Basically, giving the processor every chance to use every cycle for something meaningful. Especially on more complicated processors, the vast majority of problems I/O bound. It's basically sitting around doing nothing if you've been bad about organizing your data and algorithm to keep it fed.

But I will agree that optimizing compilers are mostly excellent these days.

[gf]

The code itself isn't slow, but the predictive branching in this case always predicts wrong

Did you try to add likely/unlikely/__builtin_expect hints to the code?
Or does your compiler not support that?

[peter-h]

Also, as optimizations go, the vast majority of the time that I've needed something optimized it was because the ALGORITHM sucked

That I am certain is true, but it is tangential to the debate.

-O0 does produce truly wasteful code. It will usually run plenty fast enough but is a lot bigger than needed. From my project notes:

The results, on the finished project (11/2024), boot block (max 32k) / main flash (1024k-32k)

-O0          15.3k / 623k
-Og          10.3k / 440k  <- used for development and production
-O1          11.5k / 493k
-Os          9.4k / 391k
-Ofast       15.5k / 490k
Others, not listed above, have not been tested.

So you can see -O0 will fill up the FLASH pretty fast and for no good reason.

[bson]

For non-trivial projects consider also concatenating all files together and compiling with -fwhole-file.  It won't work miracles, but can shave off a little bit for a production release.  More specifically it can identify unreachable code and identify inlining candidates in a call graph involving globals, and overall makes global calls (and variables) optimization non-barriers.

[xvr]

I remember my old job at Sun Microsystems (over 30 years ago). We were maintaining the SunPro compiler, and we got a bug report. At some optimization level, some user-defined function terminates with a completely wrong result. Investigation revealed that the function code contains one undefined construct (which does not intersect with the execution flow for the given input parameters). The optimizer detects the undefined construct and removes the code that depends on it. Then it removes even more dependent code, and eventually removes the entire body of the function.
It is very difficult for us to explain to the user that removing the body of his function is the expected behavior.

[John Coloccia]

I remember my old job at Sun Microsystems (over 30 years ago). We were maintaining the SunPro compiler, and we got a bug report. At some optimization level, some user-defined function terminates with a completely wrong result. Investigation revealed that the function code contains one undefined construct (which does not intersect with the execution flow for the given input parameters). The optimizer detects the undefined construct and removes the code that depends on it. Then it removes even more dependent code, and eventually removes the entire body of the function.
It is very difficult for us to explain to the user that removing the body of his function is the expected behavior.

I've seen it work the other way around too. Compiler invents code because otherwise the behavior is would be undefined....so this Must happen.  At a minimum, you're certainly allowed to infer it and assume the code is free of undefined behavior.  It's a fascinating chain of reasoning that usually does something reasonable, even when the program itself is, technically, broken. Compiler designers are wicked clever these days.

The customer is lucky he didn't end up with nasal demons...

[ejeffrey]

Is that the main purpose of using volatile?

Yes.

if we already know that compiler optimizations can cause issues in such cases, then why don’t we just turn off optimization entirely during compilation? Wouldn’t that avoid the problem?

Don't think of it in terms of optimization.

Instead think about the purpose of a program.  Some programs compute a value (return value).  Others perform IO, such as writing information to a file or a serial port, or reading from the keyboard.  Some do both.

From the perspective of a compiler, a source program is instructions that define the outputs in terms of the inputs.  Those are the observable behavior of the program.  Everything else is the implementation.

Declaring a variable volatile defines read and write operations on that variable as observable behavior, rather than implementation details.

Just like when you write:

Code: [Select]

printf("hello, world.\n");

You don't care about where the string constant is stored, how the function is called, but you do care that the output isn't "world. hello,"  Volatile declares that the specific variable reads and writes need to occur in the order written because they are observable.

That's all.

[John Coloccia]

The code itself isn't slow, but the predictive branching in this case always predicts wrong

Did you try to add likely/unlikely/__builtin_expect hints to the code?
Or does your compiler not support that?

I don't remember. It was a while ago. I recall Linux added something....and then maybe c++ added something. I may have tried it or it didn't exist, and I also may have just guessed or looked at the list file, and realized what was happening.

It was a bit of a tangent from this discussion, but I tossed it out there because optimization for a while was very important to the work I did, and it was almost never a number of instructions problem.

But like I said, optimizing compilers are pretty good these days.

[Siwastaja]

As an example, I have many other cases where I've ripped out very code and memory efficient algorithms, and replaced them with brute force searches through a large array. Extremely inefficient and "wasteful" on the surface, but in practice it absolutely screams

I can totally relate to that, and am guilty to the very same pattern. If it so happens that I can't figure out a high-level algorithm for a complex problem, and it does as I'm not really a mathematician, I just write brute-force loops and, if needed, micro-optimize shit out of them. How much modern CPUs, even just microcontollers, can do nowadays when programmed in C, with some thought given to the structure of data representation in memory, is astonishing.

Recent example is running 3 million(!) simulations of battery state-of-charge, grid energy import/export and energy cost with 1 hour resolution over 24 hours, with half a million of different control strategy combinations and six different simulation scenarios (different weather / energy use forecasting). These 3 million simulations take 5 minutes on Cortex-M4 at 64MHz, or < 1 second on a desktop PC. Competitors use neural networks, and we can read on social media how they sometimes make completely random decisions.

Usually you are just minimizing some cost function, in which case you don't need to even store intermediate results. Just keep track what parameter set gives best score, and reconstruct after the iteration is done.

Is this elegant? Probably not - but brute-forcing with some state space limitation works really well for complex multi-variable problems; computers are really good at doing small repetitive calculations, given that memory footprint is small enough that significant piece of state is kept in CPU registers or L1 cache. This sort of brute-forcing also tends to parallelize well, code being very easy to multi-thread.

But you can't do that in Python or Matlab. This absolutely depends on least amount of bloat. You can do it in C; maybe a hand-crafted assembly version is 25% faster but that's not significant. But Python would be 100 to 1000 times slower and that's a no-go.

[cv007]

Here is something that a lower optimization level (in gcc) may miss (and you may not realize you are not getting it)-

https://godbolt.org/z/foe7Ga57f

in this case array bounds checking. This is the gcc C++ compiler and is usually quite strict, but in -Og and -O1 array bounds checking is not active. In -O1 you can enable it via a few options, not sure how it gets enabled in -Og but probably can be if you can figure out the combo of options needed.


I have some code now that is compiled with -Os with no errors, but if I use -O2 or -O3 will get an array bounds error. At the -Os level the compiler sees there is no actual array bounds problem or potential problem (at least not yet) where -O2 and -O3 have array bounds checking on but are not at a point where it can determine if the array bounds has or can be exceeded. In this case the code should be changed to eliminate the possibility of ever seeing this error, but even if unaware of it by using -Os, if the array is accessed out of bounds or get to a point where the compiler cannot know a value used at compile time, the compiler will start complaining. It would be better to know sooner rather than later even when the compiler was watching out for you.


Maybe its not a bad idea to run through the optimization levels once in a while just to see how the compiler responds to each. It may not do much other than show different compile sizes, but it may also reveal as the above did that there could be a problem that is either not handled or the compiler was watching out for you all this time but the code could produce a compile error later.

[SiliconWizard]

I always do most of my development and testing at the highest optimization level I'm aiming for for production.

I only (rarely) lower it to the minimum usable for debugging with GDB if needed - and I fire GDB pretty rarely and only in hard-to-track-down cases.

I have seen too many embedded devs doing what some seem to be doing here, doing most of their development at a low optimization level and "abusing" the debugger, only to get bitten late into the development process when switching to the production flags and starting to test the whole thing for real. I really advise against doing that. Just my 2 cents.

[peter-h]

You are probably a very good programmer.

I've been doing code since the 1970s but still like to step through some stuff to make sure it is really doing what it seems to be doing. So much hardware interaction can work by luck...

[brucehoult]

You are probably a very good programmer.

I've been doing code since the 1970s but still like to step through some stuff to make sure it is really doing what it seems to be doing. So much hardware interaction can work by luck...

The problem with hardware interaction is it's often timing dependent and you CAN'T step through it -- it might be ok to delay 100 µs (to do a printf) instead of 1 µs, but 10 seconds is right out.

[Siwastaja]

I remember my old job at Sun Microsystems (over 30 years ago). We were maintaining the SunPro compiler, and we got a bug report. At some optimization level, some user-defined function terminates with a completely wrong result. Investigation revealed that the function code contains one undefined construct (which does not intersect with the execution flow for the given input parameters). The optimizer detects the undefined construct and removes the code that depends on it. Then it removes even more dependent code, and eventually removes the entire body of the function.
It is very difficult for us to explain to the user that removing the body of his function is the expected behavior.

When large parts of code disappear on UB, that I think is one of the better outcomes. Best would be, of course, a guarantee of compiler always miraculously knowing the true intent of programmer, but since that is impossible, the second best is compiler detecting UB and printing a warning - which is what compilers are much better at today. Third best is a large and obvious difference against programmer's intent, such as removal of the whole function. It will cause major grieve the first time a C beginner has not even heard about existence of UB and can't understand why compiler removed the code, but after you have seen it once, you'll recognize the pattern quickly.

It is hard to interact with people who A) are your customers so paid money to you, B) think they are right and experienced, C) really are not that experienced, D) are clearly wrong. This is a much more generic issue not limited to just compiler optimizations; just need to cope with this.

[Siwastaja]

You are probably a very good programmer.

I've been doing code since the 1970s but still like to step through some stuff to make sure it is really doing what it seems to be doing. So much hardware interaction can work by luck...

You are overestimating geniusness of others.

No matter how long you have been programming or how much code you have written, it's entirely normal you need to "step through" stuff, test with different inputs, see what outputs are produced, and scratch your head, "why?" - this is the hard work of programming. There probably are some true geniuses who just "see" the correct code in their minds but it would be 1 in a million or less.

It's just that stepping through code in debugger is an extremely inefficient process, especially on microcontrollers. There seems to be some magical appeal to it, since some people start to resort to that as their default modus operandi. I feel sorry for them.

Instead, copy-paste your iffy part to a fresh file and compile for PC (even better if your modules are small so that you can just add a testbench file and link it with the DUT - but don't worry if that's not the case). Generate inputs, and fprintf internal state and outputs into CSV file or similar. You can run on a lot of synthetic inputs you can modify ad hoc, instead of relying in real-world inputs on microcontroller. And investigate results all at once in spreadsheet software, or whatever you like best.

It may take 15 extra minutes to "prepare" for this kind of test - maybe even hours for a complicated one -, but then you can be sure what you are doing "for good". Bonus points: save it in tests/ directory and you (and others) can refer to it later if questions arise how the logic works.

[peter-h]

The problem with hardware interaction is it's often timing dependent and you CAN'T step through it -- it might be ok to delay 100 µs (to do a printf) instead of 1 µs, but 10 seconds is right out.

Of course. But you get some hilarious cases like USB where a 1 second hold-up doesn't blow it up

For text output I find the SWV ITM console output is very fast - about 10us per char.

Instead, copy-paste your iffy part to a fresh file and compile for PC

That's ok for testing algorithms, or even graphics (the Ramtex lib I use has a PC interface mode, USB connected I believe, but I've never used it because the issues I came across were mostly performance related) but mostly I don't find it useful. I do have a win32 build of MbedTLS (used in my last product) which was done by a colleague who knows x509 and which is pretty close to the embedded one and has been very handy for working out why some server won't talk.

[tggzzz]

You are probably a very good programmer.

I've been doing code since the 1970s but still like to step through some stuff to make sure it is really doing what it seems to be doing. So much hardware interaction can work by luck...

No surprises here...

I like to spilt code into two parts: hardware specific, and hardware agnostic algorithm. Tie the two together with events for hardware->algorithm, and actions for algorithm->hardware. Then two separate test harnesses can unit test (1) the hardware on its own, and (2) the way the algorithm behaves in response to the hardware and controls the hardware. When trying to understand what the hardware is doing, (1) can be driven from the keyboard.

Single stepping each on its own is practical and valuable, but trying to single step all simultaneously is likely to lead to timing problems.

[John Coloccia]

I rarely use a debugger, but it doesn't mean I don't debug. It's just a personal preference. Most guys on my team use the debugger a lot. Honestly, though, we write our code so simply and un-cleverly that day to day stuff just mostly kind of works right out of the gate. Most stuff is either obviously correct, or if it's giving you trouble you ask for a second set of eyes and in 2 minutes they find something that's obviously incorrect.

Getting interfaces going with peripherals is always a pain. That almost never works easily, and we attack it however we can. We spent a long time getting some single wire cryptographic EEPROMS working and playing with each other.  Debuggers, scopes, logging, lots of cursing...probably some beer involved too. 

[brucehoult]

Getting interfaces going with peripherals is always a pain. That almost never works easily, and we attack it however we can. We spent a long time getting some single wire cryptographic EEPROMS working and playing with each other.  Debuggers, scopes, logging, lots of cursing...probably some beer involved too.

The thing is, you write that in a single small function (or two or three) in a small test program. You don't write thousands of lines of code with that as just part of it.

Then when you get the function working properly you put it in your real program and never fiddle with it again.

[Siwastaja]

That's ok for testing algorithms

Or internal logic - everything except what happens inside peripherals. But you sure can test stuff that is close to peripherals, and should do that.

All those iffy if-elses you have operating on some real-world inputs you easily cannot observe, and which are pain in the ass to produce on real HW.

Last time I was becoming mad with my frequency counter code which had some conditions to filter out spurious edges outside expected frequency band. All really simple stuff, like take a timestamp when rising edge is detected, calculate dt between timestamps etc... Except with all tiny little real world details added (hold-off) and me not being a genius, it was PITA to see when this detection "algorithm" loses real edges.

You can just say "oh this is close to hardware so cannot be tested" and give up, but exactly this is the case where testing is most valuable, since trying to get a real world mains frequency measurement produce those desired spikes and log them takes infinite amount of time. It is untestable on lab table and debugger.

Instead, I wrote a small program which runs a counter and "generates" those "edges" - i.e., loop and call my function every time loop variable exceeds some value. Throw in some rand() to generate extra spikes (extra calls to the function), and another rand() for missing spikes. Log the indicated frequency to CSV. Calculate RMS error between the ideal input and "measured" output (of your "algorithm"). Adjust, run hours worth of synthetic test spikes in a second and see if RMS error went down. Look at the CSV output directly when investigating why.

No microcontroller needed to test. This part of code is agnostic to which ADC is actually producing the input values.

Yes, I ended up spending two days with it, but it allowed me to fix the relatively simple filtration code (a few lines of code) to a different, but similarly simple code, which performed like 100 000 times better or so. The original was a brainfart after all.

[Nominal Animal]

Instead, copy-paste your iffy part to a fresh file and compile for PC (even better if your modules are small so that you can just add a testbench file and link it with the DUT - but don't worry if that's not the case). Generate inputs, and fprintf internal state and outputs into CSV file or similar. You can run on a lot of synthetic inputs you can modify ad hoc, instead of relying in real-world inputs on microcontroller. And investigate results all at once in spreadsheet software, or whatever you like best.

I personally do this for every key algorithm and approach I do, saving each one in their own directory, with an explanatory README text file (and an executable that when run with -h, will output usage including a short summary of what the executable does).  I have literally thousands of these by now.
I don't write those in library form, but in a "fully featured" form, which can be pared down or refactored to what is appropriate for each use case.  I often end up taking a look at the documentation, then the code, and then write an algorithmically optimized version with its own test cases, and only after testing, integrate it with the larger whole application/firmware, followed by more testing.

Working with security-sensitive stuff has also "trained" me to switch viewpoints enough so that when I write tests, I can switch to thinking about the code from an adversarial viewpoint, instead of writing the tests to confirm my assumptions as the author of the code.  I've caught many a off-by-one and pathological mishandling cases that way, too.

As evidenced by this thread, I personally am definitely no genius; I'm just a good example of how diligent work can get one to the same end point.

As I said before, my default GCC/Clang/ICC settings are -Wall -O2.  For optimization, I learned over two decades ago to not try optimizing even the key parts perfectly.  Instead, I want my code to behave acceptably even for surprisingly pessimistic and pathological inputs.  So, I'm not making the best parts as good as possible, but making even the worst parts behave acceptably (for what the code is and does).  It can still look like "perfectionism" for a casual observer, but I assure you, it made a huge difference in my development workflow: there is always parts to be enhanced/refactored, but the diminishing returns of that become easier to perceive.

While I tend to use gdb less and less, I find it assuring that I can use it if necessary (even implementing my own Python accessors).  For any kind of tree and graph manipulation, or pointer-based structures, I very often implement a test case that generates pathological or random structures, and outputs their structure in Graphviz DOT format.

Put simply, I've drifted from using a debugger, to exporting the key internal state in easily accessible and visualizable structures.  Some call it "old printf style debugging", but because I do it also to the test cases, I do believe it helps me grasp the algorithms and approaches more thoroughly, and in a more intuitive manner.  Plus, it does fit into the good ol' Unix Philosophy, of joining small modular tools to accomplish tasks, instead of using one monolithic tool for everything.

[John Coloccia]

That's ok for testing algorithms

Or internal logic - everything except what happens inside peripherals. But you sure can test stuff that is close to peripherals, and should do that.

We try to take even further. In most cases...not all...we can arrange it so that we read our inputs once at the beginning, abstract that into a structure, and set all our outputs once at the end....abstracted into a structure

Everyone gets the same view of the system, as a cohesive frame, and everyone knows exactly when the outputs are set.

Clearly not possible for all problems, or even 100‰ of any problem, but it's our goal.

So the vast majority of the system just doesn't really interact with hardware at all. It's all very isolated, and is more or less very standard C that runs anywhere.

We do our best to shove the rest into corners.  I don't mind a bit of complicated, we'll understood, hacky nonsense in a corner or two. We get really nervous when we're dealing with complications at higher levels, and we try and figure out just what the helll we did wrong, and how to fix it.

[hans]

That's ok for testing algorithms

Instead, I wrote a small program which runs a counter and "generates" those "edges" - i.e., loop and call my function every time loop variable exceeds some value. Throw in some rand() to generate extra spikes (extra calls to the function), and another rand() for missing spikes. Log the indicated frequency to CSV. Calculate RMS error between the ideal input and "measured" output (of your "algorithm"). Adjust, run hours worth of synthetic test spikes in a second and see if RMS error went down. Look at the CSV output directly when investigating why.

No microcontroller needed to test. This part of code is agnostic to which ADC is actually producing the input values.

Yes, I ended up spending two days with it, but it allowed me to fix the relatively simple filtration code (a few lines of code) to a different, but similarly simple code, which performed like 100 000 times better or so. The original was a brainfart after all.

Doesn't have to be this time costly. I have a GoogleTest CMake project secondary to my embedded code. It basically includes all source except for board BSP, and in addition, I also include all *_test.cpp files into the executable. Its really easy to switch between embedded and unittest environment.

The case you're making is indeed excellent. The amount of code with hardware can often be made really really thin. If you just pass these register input/outputs around between higher level application code, then that is something you can test independent of hardware. It requires a tiny amount of extra code organization to seperate the pure hardware code from unittestable application code.

The frustrating thing with debuggers is, they are so slow and clumsy to use. First they are mostly useful on lower optimization levels. Chances I have to recompile my whole project. Then what you can find is that a function call fails, but only on call 6. So you set a breakpoint, wait for it to hit 5 times (and click run 5 times).. but then this doesn't happen because the timing of your program has changed. Aargh so you wish you had some conditional breakpoints, but this is embedded, which means you don't. Instead you go in, add some messy global/static counter value and then increment its value on each call. Then if (callcount==6) { /* Breakpoint */ }. If you have a macro that calls the bkpt instruction on ARM great, but if you forget this exists you will likely write some volatile code statement which doesn't get optimized away.
Okay so finally you can start single stepping and inspecting values inside your code. Presumably.. all this messing around took 5 to 10 minutes. And best of all, later on your forget you added this temporary callcount hack and it accidentally gets pushed to git.

I'd much rather make a short trace which contain the function arguments and return values of the application layer calls. I can shove those into a unit test (just like you're generating fake edges) and test forever. Best of all: this code has now become an automated test enviroment, to prevent regressions.

One could even test the arrival of certain events (e.g. from IRQ) on a larger application, and run a small integration test on your PC. Which then again takes over the work from manually pressing buttons on dev boards over and over to verify a certain scenario is still operational. We're programmers, so this is like automating ourselves out of the test equation.

I done this kind of testing with RF DSP algorithms where you feed ADC data to symbol synchronizers, matched filters, demodulators, etc. Its easy to add a whole bunch of noise and test at what SNR or Eb/N0 the code breaks down. Having to do this on real hardware is a pure pain. Some people may do this kind of testing in MATLAB, but then you have to bridge code translation issues.