Disabling Compiler Optimization vs Using volatile

[ejeffrey]

You should just repeat that each time you propose changing priorities to ensure you don't have a high priority interrupt depending on a low priority one

That sounds good in theory, and if there is some formal process you are required to go through to make proposals/changes, then I guess it would be taken care of.

I see it more along these lines- a 'time' isr that is updating your system time is created which of course has an isr which updates var(s), something like the systick timer. You give it the highest priority since its small, and was your only reason for using the high priority since calling the time function which makes use of the vars will account for any missed irq (as timer keeps running no matter what). So, you think about it and conclude since the highest priority is being used there is no need to protect updating the vars inside the isr. All good.

But why are you using different priorities at all?

Without a concrete reason why interrupt A has to have higher priority than interrupt B, the only sane thing to do is have them be the same priority.   Once you have a concrete reason, it's extremely unlikely that it is OK for the lower priority interrupt to block the higher priority one. 

To be clear, by priority I mean the ability for a high priority interrupt to interrupt a lower priority.  There is also the ability to set the order in which multiple pending interrupts are serviced, but that doesn't affect any of this.

I'm not sure what the downside is in allowing for different priorities to be used, without the need to change or look over the code each time. Different priorities were already accounted for when it was created, and the cost was minimal in terms of code and cpu cycles.

The point is, that isn't what you are doing. In the scenario you have described, reprogramming the interrupt controller to change the "priority" is just a placebo.  The priority is set by your use of disabling interrupts.

That sounds good in theory, and if there is some formal process you are required to go through to make proposals/changes, then I guess it would be taken care of.

The process is: "I am changing the priority of interrupt foo.  What global variables does it access, and what other interrupt handlers also access those"  If you aren't doing that then you absolutely shouldn't not be messing with it.

[JPortici]

My attitude is there should be only three priorities: normal processing, (short) ISR, and panic shutdown.

good for you!
In my target application there is the highest priority to some serial bus i have to bitbang, then that i have to deocde, then some DMA'd peripheral that i have to process BEFORE another DMA'd event comes in or the cycle breaks, then the other DMA'd peripherals, then the timer interrupt, then the COMMs' interrupt, then the "task" interrupts (which are not short at all, they just need to run over the main thread which does stuff that i don't care if it's delayed by tens of ms of other processing) and then the main thread.

I get by with 8 levels of interrupts

Without a concrete reason why interrupt A has to have higher priority than interrupt B, the only sane thing to do is have them be the same priority.   Once you have a concrete reason, it's extremely unlikely that it is OK for the lower priority interrupt to block the higher priority one. 

Exactly. in my case i have a very good reason to keep interrupts at different priorities, very short and frequent events that must be processed before anything else. And i never have to change the priorities after that.

[tggzzz]

My attitude is there should be only three priorities: normal processing, (short) ISR, and panic shutdown.

good for you!
In my target application there is the highest priority to some serial bus i have to bitbang, then that i have to deocde, then some DMA'd peripheral that i have to process BEFORE another DMA'd event comes in or the cycle breaks, then the other DMA'd peripherals, then the timer interrupt, then the COMMs' interrupt, then the "task" interrupts (which are not short at all, they just need to run over the main thread which does stuff that i don't care if it's delayed by tens of ms of other processing) and then the main thread.

I get by with 8 levels of interrupts

How on earth do you guarantee by design that you meet all those requirements? Or do you just cross your fingers and hope a rare timing miss doesn't cause a product failure that in turn causes an employment failure?

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

Failing that, a little hardware support would remove the tightest timing constraints.

Interrupts and priorities are a bodge that was useful when processors were limited and expensive. Since neither is true now, you can dispense with priorities and interrupts.

[cv007]

But why are you using different priorities at all?

Without a concrete reason why interrupt A has to have higher priority than interrupt B, the only sane thing to do is have them be the same priority. 

Maybe I'm not using different priorities today, maybe I will later, and maybe there will be a good reason, or maybe the reason will not be good. I simply protect something needing protection so whether priorities get changed or not, it needs no additional thought later. I can apply this same 'rule' consistently to all isr's with anything that needs protection. Maybe there are only a few of these around, or maybe there are quite a few, it doesn't matter as they all get the same treatment.

If protection is not needed due to it being the highest priority or all irq's are the same priority, the cost of the unneeded protection is simply an extra 3 instructions. If a higher priority irq is blocked for the time protection is active, and it had no need of the var being protected, then that irq was delayed for a dozen cpu cycles unnecessarily. If a higher priority irq is blocked for the time protection is active, and it DID have a need to use the protected var then that protection was necessary and useful. A minimal cost to never have to deal with it again. The code, such as for a systick timer, can move over to any number of cortex mcu's and it will never be subject to the decisions being made about irq priorities. There will be no notes in the code that will specify it needs to be highest priority, because it simply doesn't matter. Instead of checking your code for these problems you may have left for yourself if/when changing priorities, you can instead just focus on whether your priority change was useful or not.

example- are these interrupt locks necessary? maybe not in someIsr() and maybe not in time() as I have not said what priority someIsr() is using nor did I specify from where time() may be called. But why pre-determine at what priority time() can be safely called and who will be calling it, and also why pre-determine what irq priority someIsr() will be in relation to all the others? The protection is simple and of such little cost, it seems to me it will be worth doing so you never have to revisit this particular potential problem ever again. You figured out a var could need interrupt protection for both the write and read and took care of it. It will always be correct from day 1, and a future me cannot screw it up later.

static volatile u64 ms;
void someIsr(){
    InterruptLock lock;
    ms++;
    }
u64 time(){
    InterruptLock lock;
    return ms;
    }

These class interrupt protections can also be put in a scope, which you would do in a larger function to keep the protection only where its needed. The class constructor saves the irq state at the point of object definition, the destructor restores irq state at the end of scope.


Another example would be something like stm32 gpio registers. Some of them are shared among pins on the same port. If you want to turn on a pin pullup, or reconfigure a pin, or set it to analog, or whatever else that will modify these shared registers- do you want to pre-determine in what run level you can safely do those things? do you restrict all code that touches these registers to the same run level? do you want to restrict all pin config to a single point in time? why not simply interrupt protect any modifications to these registers so one no longer has to worry about whether some isr is changing a pin while some lower run level was also changing a pin on the same port. You no longer have a restriction about where you can reconfigure a pin- it can be done anywhere and you no longer have a potential problem. Whether you setup all pins in a single place once, or let any code setup or modify a pin, it will always work correctly and there will be no possibility one of these register modifications will be interrupted where another run level may be wanting to modify the same register. I think this is also worth doing simply because it eliminates a potential problem that never has to dealt with or thought about ever again,  also with very little cost.

I have looked around to see who also interrupt protects these gpio registers, and found that NuttX does, not sure about Zephyr as I cannot read what they are doing, and I doubt any ST HAL code is doing so. If NuttX is doing the same, at least is suggests my thinking is correct.

I'm not interrupt protecting everything in sight, nor am I declaring everything volatile. The protection is done when it is needed or may be needed as previously stated, the volatile is done when needed, after some thought and after checking the compiler output (and in -Os so no false assurances). I think I am acting sanely, but if not at least whatever I'm doing is working out well for me.

[Siwastaja]

How on earth do you guarantee by design that you meet all those requirements? Or do you just cross your fingers and hope a rare timing miss doesn't cause a product failure that in turn causes an employment failure?

Just like airplanes and space rockets are designed with extensive practical testing, and not mathematical proof alone, since human kind is still incapable of fully relying on latter, yes, we measure the performance, add safety margins, and understand the order of magnitude of potential edge cases and whether they are relevant or not.

We rarely rely on whole concepts from single manufacturers, with single point of failure, process-wise, no matter how good they sound on paper, and no matter how good marketing shills they employ. We understand the risk of vendor lock-in.

This works brilliantly well in the real world, especially with microcontrollers with no caches and no TLBs, and just a few clock cycles of timing jitter, as we have repeatedly discussed.

For really critical work, worst case timings can be formally analyzed. Register transfer level simulation models are available from ARM, for example.

Besides, I have sad news for you. How the XMOS silicon is synthesized from HDL is based on timing estimates based on decades of practical research and testing, just like design of any other IC. There is no guarantee, in your terms, that the XMOS won't just internally enter metastability. The quality is just tested into the product. Your nice "guarantees" all ride on top of this kind of handwaving on which the real world is fully based on.

[JPortici]

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

No, i want a widely available architecture, a good conforming c compiler and well unerstood peripherals
please stop the propaganda

I'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.
Guess what

[John Coloccia]

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

No, i want a widely available architecture, a good conforming c compiler and well unerstood peripherals
please stop the propaganda

I'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.
Guess what

There are some really strange ideas about C and processors kicking around. 

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project. That doesn't mean you don't need to appropriately still use "volatile" where required. They're unrelated to each other. Having optimizations turned on just makes it more likely that sloppy volatile usage will cause problems, but having them off doesn't make it unescessary.

[JPortici]

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

No, i want a widely available architecture, a good conforming c compiler and well unerstood peripherals
please stop the propaganda

I'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.
Guess what

There are some really strange ideas about C and processors kicking around. 

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project. That doesn't mean you don't need to appropriately still use "volatile" where required. They're unrelated to each other. Having optimizations turned on just makes it more likely that sloppy volatile usage will cause problems, but having them off doesn't make it unescessary.

It's a bad practice, though.
I get that writing good C can be hard, and i remember how "bad" was my writing 10 years ago. I also remember how iffy were compilers for embedded targets 10 years ago, when your advice made sense. From time to time i still find the occasional compiler bug, but more often than not it was me that was doing something wrong.
Optimizers these days are crazy good, provided you write correct code (use the appropriate types. Don't use unsigned char if you mean uint8_t, don't use unsigned int if you mean intptr_t or size_t, the compiler can perform better when you convey the correct meaning. Don't inline assembly, use assembly modules and write functions that conform to the ABI and call them from C.)
the gains in code size and execution speed you can get from -O0 to -O1 can be massive, depending on the architecture of course

[Siwastaja]

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project.

I think this has happened on every real project I have done in last 10 years. -O0 produces extremely crappy code. Size alone is a problem; I don't like to pay premium by getting the overly large flash just because I would like to compile at -O0, which has no upsides at all. I routinely end up spending at least half of the flash.

There really is no real reason for -O0 to even exist. The world would probably be a better place without it, because then people could not give bad advice like this.

Really, just use -Os or -O2 or -O1 or -Og (if you prefer to use debuggers) by default. Do not let your friends compile with -O0.

[John Coloccia]

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

No, i want a widely available architecture, a good conforming c compiler and well unerstood peripherals
please stop the propaganda

I'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.
Guess what

There are some really strange ideas about C and processors kicking around. 

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project. That doesn't mean you don't need to appropriately still use "volatile" where required. They're unrelated to each other. Having optimizations turned on just makes it more likely that sloppy volatile usage will cause problems, but having them off doesn't make it unescessary.

It's a bad practice, though.
I get that writing good C can be hard, and i remember how "bad" was my writing 10 years ago. I also remember how iffy were compilers for embedded targets 10 years ago, when your advice made sense. From time to time i still find the occasional compiler bug, but more often than not it was me that was doing something wrong.
Optimizers these days are crazy good, provided you write correct code (use the appropriate types. Don't use unsigned char if you mean uint8_t, don't use unsigned int if you mean intptr_t or size_t, the compiler can perform better when you convey the correct meaning. Don't inline assembly, use assembly modules and write functions that conform to the ABI and call them from C.)
the gains in code size and execution speed you can get from -O0 to -O1 can be massive, depending on the architecture of course

What are you talking about? Who said to turn them off to fix bad code?

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project.

I think this has happened on every real project I have done in last 10 years. -O0 produces extremely crappy code. Size alone is a problem; I don't like to pay premium by getting the overly large flash just because I would like to compile at -O0, which has no upsides at all. I routinely end up spending at least half of the flash.

There really is no real reason for -O0 to even exist. The world would probably be a better place without it, because then people could not give bad advice like this.

Really, just use -Os or -O2 or -O1 or -Og (if you prefer to use debuggers) by default. Do not let your friends compile with -O0.

Maybe you're programming toasters and need to save every last penny. Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

Compile however you want. LOL....whatever.

[brucehoult]

Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

Maybe last century, or in the 1970s or something.

To a 99.9999% approximation, gcc and llvm don't have optimisation bugs. They are used by millions of people to compile billions of lines of code, with optimisation turned on.

If you think you've found a compiler bug then

1) you're wrong, it's a bug in your code

2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

[hans]

No or low optimization level produces crap assembly output. There are plenty of applications that simply wont work if your code isn't fast.

Take for example anything battery powered. Every processor cycle consumes energy. If you want to extend battery life, then make the code complete as fast as it can.

Another would be embedded graphics. Large microcontrollers can render GUIs on (half a) mexapixel displays. I have a graphics project  that takes 3-4M cycles (between O2 and O3) to perform multiple bitmap copying, colour keying and alpha blending. Thats already on the limit for 60FPS on a >500MHz ARM Cortex-M7. If I turn optimization level off it drops to <10Hz, which is complete rubbish since the GUI requires smooth animations.

USB, audio, circular DMA streams processing.. all have either hard real-time requirements or the performance can be a banner spec of the device.

You can always buy a bigger battery, a faster/larger microcontroller, but this not about penny pinching, this is about the extent it becomes impractical. Or your competitor is doing this work, so then you're making an inferior product because of these reasons.

At some point there needs to be a transition from "praying it works" to engineering with scrutiny. I understand Siwastaja's point about testing quality into a design, on the other hand, I think it doesn't beat the hard work that goes into learning how your tools work (like correct code). In software there is no guarantee that newer compiler versions wont 'break userland' by having always-on optimizations. Similar to how we're losing some features already in newer C(++) versions, such as register keyword, removing &= or |= operators on volatile types, etc.

In my 10+ years of using GCC, I've encountered a single bug in GCC. It didn't even produce bad assembly, GCC simply segfaulted entirely. I reported the bug and it got fixed. Coincidentally, the bug was later reported about half a dozen times since ST's IDE used the older GCC version at a later release which didn't include the fix yet. The bug was triggered by ST's driver code for their new DMA controller in U5 series, where it was doing some bitwise arithmetic on pointer variables.

Imo practically this goes against some advice given here: e.g. don't update compiler when it works (it wont introduce new bugs but it will keep all the existing ones), trust that 3rd party code will work since it has already been written (it may do something exotic that works in IAR but not on Keil/GCC/etc.), or [tinfoil-hat] that you need an optimization level in the first place to have bugs be introduced into the assembly output.

[ejeffrey]

Really, just use -Os or -O2 or -O1 or -Og (if you prefer to use debuggers) by default. Do not let your friends compile with -O0.

Yeah.  The main purpose of -O0 back in the day was to _compile as fast as possible_.  That was important when debugging X11 on a 486 with gcc 2.6, but it's completely irrelevant for cross compiling for a microcontroller target on a modern desktop.   The secondary purpose was to make it look better in a debugger.  Some old compilers couldn't even generate debug symbols with optimizers on, and even those that did were often confusing.  Compilers and debuggers have gotten a lot better and -Og now exists, so -O0 is now pretty much obsolete.

Maybe you're programming toasters and need to save every last penny. Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

No, it hasn't.   That has _never_ been general advice, for as long as optimizing compilers have existed.  The "compile without optimization because it's potentially risky" mentality only seriously exists in a small corner of embedded development where people have been burned by misuse of volatile and don't really understand why.  It doesn't really exist in the broader world of software development.  Also, outside of embedded development, if you are using C or C++ you care about performance.  If you didn't care about performance you would use a different language.

In fact, while I think this is less true today, it used to be that you really wanted to always compile with optimization on because a lot of very important warnings were only generated based on analysis done by the optimizer.  So if you didn't use optimizations, you were giving up a powerful tool to find bugs before they happened.

[Siwastaja]

Yeah.  The main purpose of -O0 back in the day was to _compile as fast as possible_.

Yes. This, this and this! Being too young myself, I just barely remember this (from newsgroups discussions etc.) from around the turn of the century. Optimization settings existed because compiling C code was slow. Heck, back then one of the main lines of criticism of C was that the syntactical design choices make it slow to compile - Pascal was suggested as something that would "compile faster".

This issue has just totally disappeared - nowadays C is fast to compile, even on the highest optimization settings.

And because it does not matter anymore, many others have completely forgotten all of this discussion - or some are too young (say, started programming in 2000's) so that they never saw it. The logical outcome of how human psychology works out is they "figure out" new explanations why optimization level settings like -O0 must exist.

And sure enough, there are valid reasons - choosing e.g. between -Os and -O2 and -O3 comes to fine-tuning, and fine-tuning sometimes matters. E.g., you might want to compile some timing-critical module for speed, and all the rest for size.

But it is worth noting that the "original" idea of choosing between -O0, -O1 and -O2 was fast, slower and even slower compilation time. Now, since even -O3 compiles fast enough, there really is no reason for -O0. At least I have not figured out any, and some presented on these forums are fallacies.

[Siwastaja]

In fact, while I think this is less true today, it used to be that you really wanted to always compile with optimization on because a lot of very important warnings were only generated based on analysis done by the optimizer.  So if you didn't use optimizations, you were giving up a powerful tool to find bugs before they happened.

I think this is still very relevant today. GCC has gotten way better in static analysis, detecting UB, and warning about it. It is easy to see that some of these warnings simply go away with -O0: often such that it "seemingly works", hiding both the bug, and the warning about the compiler statically finding the bug during compile - the best thing that can happen!

Another aspect: if higher optimization level causes a difficult bug to surface (e.g., lack of volatile qualifier or lack of atomicity), that's a good thing - even if it takes a lot of manual debugging work to find the bug. The last thing you should do is to turn off the optimizer, hide the bug, and hope it's not there. I mean, what you are going to do? Manually inspect assembly to verify that turning off the optimization indeed did remove the bug, then make a comment about it and warn yourself/others from ever updating the compiler? For this to be possible, you would need to know where the bug is. In which case, you could just fix it, e.g. by adding the volatile qualifier - which makes your code both correct, and as efficient as possible (by enabling optimizations). And if you do not know where the bug is, the last thing you want is to adjust compiler setting so that it seems to disappear. It is just too likely it's still there.

TLDR: -Os -Wall -Werror. Let the compiler be your friend. They are really well designed tools.

[Siwastaja]

I understand Siwastaja's point about testing quality into a design, on the other hand, I think it doesn't beat the hard work that goes into learning how your tools work (like correct code).

Of course. My comment was tongue-in-cheek reference to tggzzz's recurring comments that quality cannot be tested into a product, which is easy to agree with. It's just that in actual real world, in every real project (when looked as a whole, not just a small subset), some quality always ends up being tested into the product; handwaving will be applied, practical measurements will be used instead of mathematical proof.

Nothing indeed beats the hard work of understanding the theory, formulate the whole system as equations, and prove that the system performs to specifications. But human kind is far from capable of applying such pattern to everything; this still is possible only in very limited scope.

tggzzz prefers to use a CPU where instruction timing can be proven to accuracy of single cycle; yet he must accept that it runs on a silicon where propagation delays are hand-waved with decades of practical experience and formulae with ad-hoc coefficients, and manufactured on dies with process variations and measurement and binning of the resulting chips, all with ad hoc safety margins. And he lives in a building where all structural dimensioning is based on centuries long practical experience of what typical materials can carry and when they collapse, with generous safety margins added.

The fact that measuring ISR timing on a single-core microcontroller CPU with an oscilloscope, understanding that the subtleties like flash prefetch or branch prediction won't be realistically making a big difference at all, then applying a safety margin, does not differ at all from all other engineering work applied to all other fields - safety critical or not.

And quite funnily, really the biggest difference between general purpose and safety critical engineering is in testing.

But yeah. Of course I agree with tggzzz and you: the #1 priority is to try to understand, model, calculate and prove. It is just not always possible - sometimes not at all, sometimes theoretically possible but prohibitively expensive.

[JPortici]

What you really want and need is 8 small simple cores, one for each of those tasks. No interrupts, no cache, just guaranteed performance.

No, i want a widely available architecture, a good conforming c compiler and well unerstood peripherals
please stop the propaganda

I'd probably need 32-40 cores XMOS to do everything (counting by the peripherals / function blocks), or keep using a dsPIC and still have margin to add new functionality.
Guess what

There are some really strange ideas about C and processors kicking around. 

As to the original question, just turn all of your optimizations off, and leave them off. It's very standard practice to do no optimizations unless you run into a specific need for it. I've yet to run into such a need on a real embedded project. That doesn't mean you don't need to appropriately still use "volatile" where required. They're unrelated to each other. Having optimizations turned on just makes it more likely that sloppy volatile usage will cause problems, but having them off doesn't make it unescessary.

It's a bad practice, though.
I get that writing good C can be hard, and i remember how "bad" was my writing 10 years ago. I also remember how iffy were compilers for embedded targets 10 years ago, when your advice made sense. From time to time i still find the occasional compiler bug, but more often than not it was me that was doing something wrong.
Optimizers these days are crazy good, provided you write correct code (use the appropriate types. Don't use unsigned char if you mean uint8_t, don't use unsigned int if you mean intptr_t or size_t, the compiler can perform better when you convey the correct meaning. Don't inline assembly, use assembly modules and write functions that conform to the ABI and call them from C.)
the gains in code size and execution speed you can get from -O0 to -O1 can be massive, depending on the architecture of course

What are you talking about? Who said to turn them off to fix bad code?

The premise of this whole thread? Also, it's what i picked up from your post, which is what i've read online from very old posts, or mediocre programmers (not assuming you are the latter, but the former: a person with long time experience, that is maybe fixed in the old way of doing things.)

[JPortici]

Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

Maybe last century, or in the 1970s or something.

To a 99.9999% approximation, gcc and llvm don't have optimisation bugs. They are used by millions of people to compile billions of lines of code, with optimisation turned on.

If you think you've found a compiler bug then

1) you're wrong, it's a bug in your code

2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

3) you are compiling for an unusual target XC16 doesn't have the wave of programmers you have for X86 / ARM, the change you find one are higher

[brucehoult]

Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

Maybe last century, or in the 1970s or something.

To a 99.9999% approximation, gcc and llvm don't have optimisation bugs. They are used by millions of people to compile billions of lines of code, with optimisation turned on.

If you think you've found a compiler bug then

1) you're wrong, it's a bug in your code

2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

3) you are compiling for an unusual target XC16 doesn't have the wave of programmers you have for X86 / ARM, the change you find one are higher

Sure, a boutique compiler for a weird ISA, probably maintained by a team of 2 or 3 people, is more likely to have optimisation bugs.

AVR has a GCC port, so shouldn't have any more optimisation bugs than any other GCC target.  Some madman has recently done a 6502 back-end for LLVM, so ditto there.

But .. never having seen or used it .. I thought XC16 was based on GCC also?

In which case you might expect the odd code generation bug, but not language conformance or optimisation.

[JPortici]

Across time and space the general advice has always been to compile without optimizations unless you need them. It makes debugging and object code coverage more difficult, and it exposes you to more compiler bugs.

Maybe last century, or in the 1970s or something.

To a 99.9999% approximation, gcc and llvm don't have optimisation bugs. They are used by millions of people to compile billions of lines of code, with optimisation turned on.

If you think you've found a compiler bug then

1) you're wrong, it's a bug in your code

2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

3) you are compiling for an unusual target XC16 doesn't have the wave of programmers you have for X86 / ARM, the change you find one are higher

Sure, a boutique compiler for a weird ISA, probably maintained by a team of 2 or 3 people, is more likely to have optimisation bugs.

AVR has a GCC port, so shouldn't have any more optimisation bugs than any other GCC target.  Some madman has recently done a 6502 back-end for LLVM, so ditto there.

But .. never having seen or used it .. I thought XC16 was based on GCC also?

In which case you might expect the odd code generation bug, but not language conformance or optimisation.

Yes, bugs that happened on code generation side, but that came up with optimizations enabled. One of my favourite is when it forgot all the addressing modes and kept calculating all addresses by hand instead of just using indirect load/store, which shouldn't have done in -O0 anyway (and it took me a long time to make the zombie at first level support understand that) or another in which a particular pattern created bad opcodes (an updated device added bitfield insertion/extraction instructions, i somehow managet to write a structure type that also used bitfields which broke the instruction, but only when optimizing because in -O0 it wouldn't use bitfield instructions, a slight change in the structure would make the problem go away)

all usually patched within days

I'm trying to remember if i ever caught a bug in language conformance. Probably, but only once - and i'm pretty sure that was in XC8 which at the time was not GCC nor LLVM so it doesn't apply

[newbrain]

If you think you've found a compiler bug then

1) you're wrong, it's a bug in your code

2) if somehow you really have ... which is EXTREMELY unusual ... then report it and it will get fixed.

Agree on all counts.

As for 2) they may take take their good time though!
https://github.com/llvm/llvm-project/issues/54014
Written on clang 13, still there in 20...Not the highest priority, granted. I'll bump it, maybe.

As always I start from the assumption that a PEBKAC, so I had a hard time understanding it was a compiler bug.
Records in an I²C E²PROM were randomly not being saved or reappearing after being marked as deleted, the compiler was at the bottom of the suspects list.

This is one of the cases when a debugger was useful - I could clearly see the write in the C code happening, with the assembly doing nothing!

[brucehoult]

As for 2) they may take take their good time though!
https://github.com/llvm/llvm-project/issues/54014
Written on clang 13, still there in 20...Not the highest priority, granted. I'll bump it, maybe.

Ohh nasty! Reproduced on RISC-V, as you'd expect. It seems to be a Clang bug not LLVM optimiser or back end.

Even worse, maybe, in Clang adding the following function...

Code: [Select]

int g(void) {
    return block->counter;
}

... it simply returns a constant 255 regardless of the optimisation level.

The store to the memory location in f() is omitted at higher than -O0.

GCC always does the right thing, and every optimisation level, with g() loading the value from memory.

If g() does not exist then GCC omits the store to memory in f() at O1 or higher, and indeed omits the variable block entirely, but this is ok because it returns the new value, 252, and no one else can see the modified value.

The bug in Clang with optimisation on is not that it omits the store, but that it returns the initial value not the modified value.

[Siwastaja]

Across time and space the general advice has always been to compile without optimizations unless you need them.

I realized what John Coloccia's misunderstanding here probably is; indeed I have never heard this "general advice" anywhere, but instead, I have heard many times that "premature optimization is the root of evil", and similar remarks. Which is generally useful idea; correct to an extent. Maybe John Coloccia misunderstood this advice and thought it means compiler optimization settings, but that's really not the point. We are advised against premature manual (micro-)optimization because it easily leads to harder-to-understand, harder-to-maintain code, and also because knowing what the actual effects are is surprisingly difficult.

Instead, really the general advice is to try to write as high-level code as possible, designed for understanding, maintainability, correctness and error handling, and then let the compiler figure out how to micro-optimize it.

Don't take that thinking too far, though - this principle only applies to manual micro-optimization, and micro means small localized adjustments. Choosing efficient data structures and algorithms is sometimes of paramount importance and compilers can't do anything about it if you chose a O(n^2) sorting algorithm which takes 100x too long, or chose a memory representation of data which takes 2x the available RAM.

Also worth noting that if one enjoys manual micro-optimizations and gets them right, who are we to say it should not be done. But compiling at -O0 and then manually micro-optimizing instead would be probably worst use of one's time ever.

[SiliconWizard]

But yeah. Of course I agree with tggzzz and you: the #1 priority is to try to understand, model, calculate and prove. It is just not always possible - sometimes not at all, sometimes theoretically possible but prohibitively expensive.

And even so: formal proofs are great when doable, but they can still only prove that a system is correct according to its specifications.

Thing is, most of us know that in practice, a very significant proportion of bugs/issues are not due to a bad implementation (that's essentially a "skill" issue, and sure, some tools and methods can make it easier), but to incorrect or incomplete specifications, and the only way we can find this out is to test the system in its nominal environment, as it will be used (this step is usually called "validation"). Some studies show that as much as 80% of defects in a complex product come from defects in its specifications rather than in the implementation. No language or hardware is going to help with that.

So there's definitely a matter of perspective here. From the perspective of an engineer in charge of *implementation*, tools are more of a concern. Whatever can help them output correct implementations is welcome (and sometimes seen as a necessity). From the perspective of people in charge of the whole system, that's a different story. If you design a rocket with provably 0 software or hardware defect and it crashes on the day it's launched - because some specification on what to do when some combination of sensors reading was lacking - then it's still game over. That would mean hundreds of millions or billions lost, possibly severed reputation and in worst cases, deaths. But sure - the team in charge of the implementation will not be held responsible.

[tggzzz]

But yeah. Of course I agree with tggzzz and you: the #1 priority is to try to understand, model, calculate and prove. It is just not always possible - sometimes not at all, sometimes theoretically possible but prohibitively expensive.

And even so: formal proofs are great when doable, but they can still only prove that a system is correct according to its specifications.

Thing is, most of us know that in practice, a very significant proportion of bugs/issues are not due to a bad implementation (that's essentially a "skill" issue, and sure, some tools and methods can make it easier), but to incorrect or incomplete specifications, and the only way we can find this out is to test the system in its nominal environment, as it will be used (this step is usually called "validation"). Some studies show that as much as 80% of defects in a complex product come from defects in its specifications rather than in the implementation. No language or hardware is going to help with that.

So there's definitely a matter of perspective here. From the perspective of an engineer in charge of *implementation*, tools are more of a concern. Whatever can help them output correct implementations is welcome (and sometimes seen as a necessity). From the perspective of people in charge of the whole system, that's a different story. If you design a rocket with provably 0 software or hardware defect and it crashes on the day it's launched - because some specification on what to do when some combination of sensors reading was lacking - then it's still game over. That would mean hundreds of millions or billions lost, possibly severed reputation and in worst cases, deaths. But sure - the team in charge of the implementation will not be held responsible.

Formal proofs were very fashionable in the 80s, but almost always they ran into a showstopper: there were significant system components about which formal reasoning wasn't possible because it was too complicated.

I remember discussing possibilities with a research group working on FSMs. They were proud of their progress. We showed them the FSMs for the computer network we were implementing (FDDI, shudder!), and asked whether they could prove freedom from deadlock/livelock. They took one look and threw their hands up in horror!

Testing will always be necessary, but using good tools (of any type including brainpower) should enable whole swathes of problems to disappear. Certainly measure and hope you've spotted the worst case is highly suboptimal.