No more code-size-limited version of IAR embedded workbench for ARM?

[ejeffrey]

That being said, I'm wondering about this 'while (1)' thing. What kind of optimization could possibly optimize an infinite loop away, I don't know. An infinite loop is not something any optimization can get rid of, that I can think of right now.
Finite loops, OTOH, absolutely, if the body part of the loop has no effect.

So, unless you have a 'while (1)' loop that actually has an exit path in its body, and said path is statically determined by the compiler to be true at some iteration, while the rest of its body has no effect. Which would make it a finite loop.

There is no real reason to specifically optimize out an infinite loop.  However, it's valuable to optimize out an empty, finite loop.  For instance, a loop might have it's entire body hoisted out of the loop, resulting in an empty loop.  Furthermore, it's useful to optimize out an empty loop even if it can't be proven to terminate (this could happen due to the loop counter being a different type than the bounding expression).  To support this (and other optimizations related to concurrency) the C and C++ standards state that an infinite loop with no side effects is undefined behavior.  This allow compilers to assume that all loops with an empty body will eventually terminate and remove remove them.

Since people want to write infinite loops, or finite empty loops for time delays, compilers sometimes try to preserve these as a special case, but that always risk false positives and false negatives.  The correct way to implement this is to put a volatile access or volatile inline asm (even a nop) inside the loop body.  This will mark the loop body as observable behavior and prevent the loop from being removed.

[nctnico]

You cut off the bit that said the control for warnings should not change, so older projects still build, completely changing my meaning. To be clearer, new warnings need to have new controls, apart from a blanket "Give me all the warnings you have" option, which no long term make files should use.

I would recommend adding new warnings to use, though. They could reveal bugs that have been always there.

"It's working" argument is bullshit, usually. I mean, how do you know it is working? Maybe 10000 customers are using it, and maybe they have intermittent problems they deal with, which decreases the product quality experience, but not enough for them to make formal bug reports? Maybe they don't know how to report. Maybe they are blaming themselves for "doing something wrong" and working around the bugs?

Enabling more/better warnings from newer tools and going through the warnings is, IMHO, time well spent. If you don't have resources to do that, then, obviously, don't touch anything, don't update your toolchain.

But if you have even a little bit of extra resources to spend on software quality improvements, enabling more warnings seems like a pretty low hanging fruit.

I agree. The whole point of the additional warnings is to highlight additional potential problems, and ultimately those warnings should be addressed if the project has a long term future. The problem is when you go from a clean compile to a flood of complaints from the tools its very hard to know where to start. If you make the thousands (No exaggeration. Thousands is typically on the low side) of source code changes needed to remove the warnings from an older project on a recent tool chain you are going to make at least a few errors, and the project will be broken. You need a way to get easily get back to a clean build, so you can move forwards incrementally with the necessary changes, and test along the way.

Yep. But it doesn't hurt to fix the most pressing warnings. In these cases I do take the time to go through the warning and see if there are possible casting (size), pointer arithmetic and buffer overrun issues.

But these are rather simple things to deal with. Things get worse when warnings become hard errors in newer compiler versions. A while ago I wanted to compile an older GCC (for a legacy project) with a relatively new GCC and that didn't work out of the box. The newer GCC doesn't allow re-definition of a function which older GCCs let slide with a warning. This meant patching the old GCC version in a few places. Nothing major; just cleaning up definitions which should have been done correctly right from the start.

[5U4GB]

This is gonna be a long off topic diversion but - while I probably agree with you - IOT boxes should never be on an open port. They should always have an absolutely minimal attack surface.

Oh, absolutely.  Some of the most secure devices I've ever audited were written by embedded-systems developers (by which I mean RTOSes and bare-metal, not Linux shovelled onto a generic SoC-based device) who had little to no security experience but created implementations with almost no attack surface.  The usual quick-start of fuzzing the TCP stack or TLS implementation or whatever to justify further investigation never worked because everything just bounced off, it wasn't until we went to line-by-line checking of the code that we found a few minor and easily-remedied issues.  So you had code that definitely wasn't the best quality stuff ever created that was nevertheless almost immune to any kind of attack because there was nothing there to attack - when 20,000 lines of near-incomprehensible X.509 parsing and processing code are replaced by a memcpy() it eliminates an awful lot of attack surface.

[mark03]

So, unless you have a 'while (1)' loop that actually has an exit path in its body, and said path is statically determined by the compiler to be true at some iteration, while the rest of its body has no effect. Which would make it a finite loop.

Since people want to write infinite loops, or finite empty loops for time delays, compilers sometimes try to preserve these as a special case, but that always risk false positives and false negatives.  The correct way to implement this is to put a volatile access or volatile inline asm (even a nop) inside the loop body.  This will mark the loop body as observable behavior and prevent the loop from being removed.

Since we are completely off topic now, I may as well join in

Finite empty loops for time delays are handy while debugging other issues but I would not use them in production code.  And while debugging I would not be compiling with optimizations.

OTOH, busy-waiting on a status register bit (again with an empty loop body) is a pretty standard use case for things like clock configuration while waiting for a PLL to lock.  It wouldn't make any sense to use a fancier approach with interrupts for init code which only runs at startup.  Is the claim that an optimizing compiler *should* be able to remove such a loop?  What if the register is marked volatile in the CMSIS header file (pretty sure it is)?  If the standard says that something like "while (periph->reg & 0x20) {}" (when periph is a pointer to volatile uint32 e.g.) is undefined behavior and fair game for elimination, then I would respectfully submit that the standard needs improvement.

But I don't think this is what is being claimed...  is it??  Just trying to understand since apparently, I'm being told that even the experts are getting this wrong

[peter-h]

I have used infinite loops on register bits in unusual cases where it cannot hang unless the silicon is faulty e.g.

Code: [Select]

// Start a conversion
ADC1->SR = ~(ADC_FLAG_EOC | ADC_FLAG_OVR);
ADC1->CR2 |= (uint32_t)ADC_CR2_SWSTART;
// Wait for End of conversion flag (bit 1 in SR)
// This could HANG but only if the silicon is defective or perhaps if the ADC was not enabled
while (((ADC1->SR) & ADC_FLAG_EOC)==0) {}

Never seen these optimised away, AFAICT. But then would I know?

Yes indeed registers are all declared volatile in the .h files.

AIUI, a loop containing a volatile variable should never be optimised away... or can it?

I know you can have fun with optimisation stripping out code which then stops debugging working, so I have used e.g.
asm("nop");   
to create code on which a breakpoint can be safely set.

[SiliconWizard]

I think my earlier point was still not fully understood.

Finite loops and infinite loops have nothing to do with each other.

An infinite loop (such as 'while (1)' with no exit path inside its body, or equivalently 'for (; ; )' ) can't be optimized out, as it's blocking the execution flow, making it impossible for any compiler to optimize it out. That would be such a severe compiling bug that it would need to be thrown away.

As long as you have a loop with an exit path, this is NOT an infinite loop. But from experience and what can be assumed in this thread, there is some common misnaming of finite and infinite here. It seems that many people will call a loop with a simple condition based on a counter with a fixed range a "finite" loop, and anything else an "infinite" loop. Which isn't correct IMO. And that leads to confusions ("unfortunate shortcuts") regarding optimizations.

'while (1) {}' is an infinite loop, has no exit condition whatsoever, and can never be optimized out, as it blocks execution flow. What happens inside the body loop doesn't matter - it can't get out.

'for (int i = 0; i < 100; i++) {}' can definitely be optimized out. It has an exit condition that can be determined statically and no effect.
What confuses some developers is that execution time itself is NEVER considered an observable effect in the language specification in C, and many (if not most) other languages higher level than assembly. (Well, there are even some assemblers that can optimize instructions, so...) For embedded developers in particular, this point often sounds confusing, as timing in embedded development is key. But C and other similar or higher-level languages have no notion of timing whatsoever.

The only "right" way of writing a delay loop in C is to either use a volatile-qualified counter, such as: 'for (volatile int i = 0; i < 100; i++) {}' or do something that is considered having an effect in the body loop - which may not be trivial. When coding for MCUs, a relatively common example of this is to use some kind of 'nop' (which is usually defined as inline assembly with a 'nop' instruction for the given target, qualified volatile, so that it itself can't be optimized out - something like 'asm volatile ("nop")' ). So that would look like: 'for (int i = 0; i < 100; i++) { nop(); }'. In either case, it will still be a 'hack' when it comes to obtaining a particular delay, but it will spend some execution time for sure. For the record, the version with a volatile-qualified counter is usually the more expensive one, as it's most often implemented with a counter placed on the stack, and read and written at each iteration, which is more expensive usuallly than a typical "nop" instructtion, while the version without the volatile qualifier will usually be implemented with a register.

As to busy loops reading some MCU "register" (not to be mixed with the CPU registers themselves), yes, as long as said register is declared volatile, the loop will never be optimized out either. That's guaranteed to work, and that's the reason register definitions are all marked volatile in well-written C: all accesses are guaranteed to be honored by the compiler.

Note that with some compilers, dereferencing pointers that are cast from integer values (which can usually be considered direct "addresses" on many targets), even when not qualified volatile, does act as though it was. That may be one reason some developers have seen that not using volatile for "register definitions" works, and so using "volatile" can be done without. I wouldn't recommend that, as there is no such guarantee that I've seen in any standard revision, so you'd be playing with the particular behavior of some compiler. What I mean is for instance:

'*(uint32_t *) 0x10000' would never be optimized out, and so would be equivalent to '*(volatile uint32_t *) 0x10000'. But that would be a particular case with a particular compiler and should not be considered a rule.

The typical case where a loop is very likely to be optimized out is the following:

Code: [Select]

uint32_t n;

void Foo(void)
{
    while (n > 0) {}
}

The reason is that the compiler, here, can assume that n never changes when Foo() is being executed.

Adding the volatile qualifier to the declaration of n will prevent this optimization, and will compile as a loop which reads and compares 'n' at each iteration.

I think that does sum it up reasonably (let me know if I missed some case) and there is no black magic involved.

[JPortici]

So, unless you have a 'while (1)' loop that actually has an exit path in its body, and said path is statically determined by the compiler to be true at some iteration, while the rest of its body has no effect. Which would make it a finite loop.

Since people want to write infinite loops, or finite empty loops for time delays, compilers sometimes try to preserve these as a special case, but that always risk false positives and false negatives.  The correct way to implement this is to put a volatile access or volatile inline asm (even a nop) inside the loop body.  This will mark the loop body as observable behavior and prevent the loop from being removed.

Since we are completely off topic now, I may as well join in

Finite empty loops for time delays are handy while debugging other issues but I would not use them in production code.  And while debugging I would not be compiling with optimizations.

OTOH, busy-waiting on a status register bit (again with an empty loop body) is a pretty standard use case for things like clock configuration while waiting for a PLL to lock.  It wouldn't make any sense to use a fancier approach with interrupts for init code which only runs at startup.  Is the claim that an optimizing compiler *should* be able to remove such a loop?  What if the register is marked volatile in the CMSIS header file (pretty sure it is)?  If the standard says that something like "while (periph->reg & 0x20) {}" (when periph is a pointer to volatile uint32 e.g.) is undefined behavior and fair game for elimination, then I would respectfully submit that the standard needs improvement.

But I don't think this is what is being claimed...  is it??  Just trying to understand since apparently, I'm being told that even the experts are getting this wrong

- You may want delays for whatever reason, using code. There are compilers, or libraries, that provide delay functions a la __delay_ms(x) and you should be using those as the compiler expects those for delays.
- You should be debugging using the same optimization level as production code. Sucks when a bunch of expressions that produce bad results are optimized and you can't see step by step, but you can get around it while debugging anyway. Otherwise you debug different code than the one in production. It can be a really bad problem.
- Peripheral registers are indeed marked volatile, because they operate on "their own thread" so the compiler must never assume their value. That's why waiting on a peripheral bit is never optimized away.

[peter-h]

for (int i = 0; i < 100; i++) {}' can definitely be optimized out. It has an exit condition that can be determined statically and no effect.

How about

for (int i = 0; i < 100; i++) {asm("nop");}

That can also be determined statically, but asm should never be optimised out (unless the whole block of code is never referenced, etc, in which case the linker will dump it).

You should be debugging using the same optimization level as production code

Couldn't agree more, but lots of people differ I use -Og. -O1/2/3 produce marginal differences, on arm32.

There are compilers, or libraries, that provide delay functions a la __delay_ms(x) and you should be using those as the compiler expects those for delays.

IME, the bigger case for loops is for very short delays, ns or us. The compiler is unlikely to provide that, because it can't be done with a normal "tick".

[jfiresto]

How about

for (int i = 0; i < 100; i++) {asm("nop");}

That can also be determined statically, but asm should never be optimised out (unless the whole block of code is never referenced, etc, in which case the linker will dump it).

To be extra safe, you might want to add "volatile" to that

Code: [Select]

#define asm(text) asm volatile (text)

– for avr-gcc, at least.

[peter-h]

That I don't understand at all. A volatile NOP??

[jfiresto]

That I don't understand at all. A volatile NOP??

The volatile is meant to stop the optimizer from removing code, that from its perspective, does nothing, and raising constant code out of a loop. The avr-gcc 3.3 inline assembler cookbook mentions its use. It may not be a problem with later versions after the avr backend lost the plot.

[peter-h]

I always assumed (wrongly?) that any assembler anywhere is never optimised away.

[KE5FX]

The fact that we all have to either assume, guess, or spend years doing nothing but studying massive, ever-changing specs in order to write safe, secure, and error-free code is the problem.

It's a failure of the toolmakers to meet the needs of the market.  Instead, compiler authors pursue their own imaginary needs, trying to save every last CPU cycle at any cost, even though no embedded developer in his/her right mind relies on the compiler to do that. 

The language isn't blameless but the real problem is the mentality behind the compilers.

[coppice]

The fact that we all have to either assume, guess, or spend years doing nothing but studying massive, ever-changing specs in order to write safe, secure, and error-free code is the problem.

It's a failure of the toolmakers to meet the needs of the market.  Instead, compiler authors pursue their own imaginary needs, trying to save every last CPU cycle at any cost, even though no embedded developer in his/her right mind relies on the compiler to do that. 

The language isn't blameless but the real problem is the mentality behind the compilers.

The real problem is the language specs don't cater for a lot of corner cases. Especially ones really important for effective embedded development. The compiler developers find things that can't be expressed properly, and add ways to express them that go beyond the language spec. Things like special attributes that will get the right behaviour in interrupt routines. None of this is portable. If the tool developers move on, and new ones come in who don't fully appreciate why these things have been done, they often break them. We need languages properly defined for the needs of embedded development.

[5U4GB]

How about

for (int i = 0; i < 100; i++) {asm("nop");}

That can also be determined statically, but asm should never be optimised out (unless the whole block of code is never referenced, etc, in which case the linker will dump it).

This was very useful because gcc used to not mess with functions with any asm() parts in them***, which was a quick hack for getting around optimiser bugs.  Or, alternatively, a "fix" after you'd spend several hours trying to figure out how the compiler could possibly be generating the code it did for the C input it was getting.

*** "Used to" being the operative word, who knows what it does today.

[5U4GB]

The language isn't blameless but the real problem is the mentality behind the compilers.

This is exactly my beef with gcc.  In almost no other industry would the blame-the-user attitude of the compiler developers be tolerated.  Look at something like table saws, they have fences, sleds, riving knives, blade guards, sawstop (if you can afford it), ... .  If the gcc developers made table saws their approach would be "it was UB cutting timber that way so it's your fault.  Oh, and good luck getting those fingers reattached".

[cfbsoftware]

Instead, compiler authors pursue their own imaginary needs, trying to save every last CPU cycle at any cost, even though no embedded developer in his/her right mind relies on the compiler to do that. 

Fortunately not all compiler authors can be tarred with the same brush. Prof Wirth's followed his own advice in his Compiler Construction book when he later developed his Oberon language compiler for Arm:

Furthermore, we must distinguish between optimizations whose effects could also be obtained by a more appropriate formulation of the source program, and those where this is impossible. The first kind of optimization mainly serves the untalented or sloppy programmer, but merely burdens all the other users through the increased size and decreased speed of the compiler.

https://www.amazon.com/exec/obidos/ASIN/0201403536/acmorg-20

You can download an official copy from the ETH website:

https://people.inf.ethz.ch/wirth/CompilerConstruction/index.html

[5U4GB]

There's also a C compiler called CompCert which comes "with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program".  Unfortunately the code it produces is pretty poor, a bit like gcc -O0 the last time I checked.

[Siwastaja]

This is exactly my beef with gcc.  In almost no other industry would the blame-the-user attitude of the compiler developers be tolerated.  Look at something like table saws, they have fences, sleds, riving knives, blade guards, sawstop (if you can afford it), ... .  If the gcc developers made table saws their approach would be "it was UB cutting timber that way so it's your fault.  Oh, and good luck getting those fingers reattached".

And yet, in the same real world where GCC is used all the time without any major problems, fingers also get cut regardless of safety features - which are insufficient to prevent most accidents because you have to keep the blade exposed to allow cutting materials - or because some products are non-compliant to safety regulations due to human error or even purposeful cost-cutting. And as you mention, not everyone can afford "sawstop". Compare this to free-of-cost static analyzer tools that can easily catch UB and thus prevent GCC's tricks, all you need is the will to use these tools. Some choose to cut their fingers instead because they don't care.

During last two decades or so, GCC has introduced huge loads of useful warnings and static code analysis to catch real-world bugs in code. These warnings are equivalent to your blade guards, sawstops etc. Only in your twisted worldview, some failures to deliver exactly what users need negate all of this work, and only in your twisted worldview, occasional defects are taken as malice or extreme unsuitability for the work role.

[Siwastaja]

There's also a C compiler called CompCert which comes "with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program".

You realize this "perfect" compiler also won't be able to implement your intended meaning when you write UB?

[coppice]

During last two decades or so, GCC has introduced huge loads of useful warnings and static code analysis to catch real-world bugs in code. These warnings are equivalent to your blade guards, sawstops etc. Only in your twisted worldview, some failures to deliver exactly what users need negate all of this work, and only in your twisted worldview, occasional defects are taken as malice or extreme unsuitability for the work role.

Has it really had a positive effect, though? As the years have gone by I have spent many hours getting my long term code to compile cleanly with the latest C compilers in maximum warnings mode. Very seldom have those extra warnings picked up anything useful, and I can't remember once actually clearing out a real bug through that work. My code isn't perfect, though. The improved analysis just hasn't been hitting the actual problems that show up from time to time, and get addressed by other means. I wonder just how many fewer bugs we have because of all this extra analysis?

[5U4GB]

There's also a C compiler called CompCert which comes "with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program".

You realize this "perfect" compiler also won't be able to implement your intended meaning when you write UB?

I realise that UB seems to be some sort of pet hobby horse of yours but the major problem with gcc that CompCert doesn't have is that gcc will silently rewrite code to change its semantics, which is something that CompCert is guaranteed never to do.

In the specific case of UB though, it depends on what you count as UB.  For example the assumption that you're running on a two's-complement machine, as approximately 100% of all systems from the last half century are (I think the last one's-complement machine was the CDC 6600, which predates the existence of C) is technically UB because you could be running your C compiler on an ENIAC and therefore we can't assume two's-complement.  However any sane compiler will apply two's-complement semantics by default since that's what it's compiling for.  gcc won't, obviously, because it's gcc.

[5U4GB]

Has it really had a positive effect, though? As the years have gone by I have spent many hours getting my long term code to compile cleanly with the latest C compilers in maximum warnings mode. Very seldom have those extra warnings picked up anything useful, and I can't remember once actually clearing out a real bug through that work.

I would say it's actually gone backwards, because more recent versions will silently do things like remove null-pointer checks without emitting any warnings.  Like you, I don't think I've found anything useful in gcc warnings for a long, long time.

[SiliconWizard]

for (int i = 0; i < 100; i++) {}' can definitely be optimized out. It has an exit condition that can be determined statically and no effect.

How about

for (int i = 0; i < 100; i++) {asm("nop");}

That can also be determined statically, but asm should never be optimised out (unless the whole block of code is never referenced, etc, in which case the linker will dump it).

I mentioned that very approach in my (probably TLDR) post. Note a few things:

- Inline assembly is not part of the C standard and so, well, there is no standard behavior. The standard actually lists it as a 'common extension' but does not detail anything about it.
- So it's up to the compiler's implementation. That's typically what would be called "implementation-defined" (and not 'undefined behavior" here). Your best bet would be to read about this in its manual. Not always easy to find, but more reliable than guessing.
- Wether the compiler can do any optimization on inline assembly is of course completely implementation-defined as well. There is no guarantee that all compilers will behave similarly, even if this extension is, as the standard recognizes, a "common" one.

What can be said about GCC (which is what many use these days due to it being the #1 compiler for ARM Cortex M and now RISC-V development) is the following, found in its manual (references below):
- For inline assembly without operands (that would be the case for a typical "nop"), it's never optimized out.
- For inline assembly with "C" operands, it *can* be optilmized out depending on how the operands are used in the rest of the code. In that case, to prevent optimization, you must add the "volatile" keyword to the "asm" one.

I seem to remember that older versions of GCC could optimize out even inline assembly without operands, which is why it has become common to always use "asm volatile" to use assembly that must be inlined verbatim, even when not using operands. That also allows not having to think about it - always use volatile, and you have your guarantee. (It's not necessary again for recent versions of GCC and assembly with no operands, but it's accepted, so you can always use it.)
That's why I mentioned the 'asm volatile ("nop")'  sequence before as an example.

To understand what inline assembly is and how it works, you can refer to the links below.

https://gcc.gnu.org/onlinedocs/gcc/Basic-Asm.html
https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html

The syntax for inline assembly with operands is not trivial. Takes a while to figure out.

For other compilers, Clang (LLVM) tries to mimick GCC's behavior as much as possible (to be a drop-in replacement), so it should behave similarly for inline assembly. For other, commercial compilers, quite a few probably have a similar syntax but I would recommend reading about it in their respective manual to make sure. Compilers that are mainly for embedded targets (which GCC isn't) are likely to favor more "tamed" behavior and so not require "volatile" for any inline assembly sequence, but that would just be a general thought. That doesn't mean that they do it better, just that they do it in a way that's more likely to be what expects their intended audience. GCC and Clang/LLVM are general purpose compilers that support dozens of targets, from small MCUs to servers and supercomputers.

[peter-h]

For inline assembly with "C" operands, it *can* be optilmized out depending on how the operands are used in the rest of the code. In that case, to prevent optimization, you must add the "volatile" keyword to the "asm" one.

Does that mean that this might get optimised out?

Code: [Select]


// Hang around for delay in ms. Approximate but doesn't need interrupts etc working.

__attribute__((noinline))
static void hang_around(uint32_t delay)
{
extern uint32_t SystemCoreClock;
delay *= (SystemCoreClock/4100);

asm volatile (
"1: subs %[delay], %[delay], #1 \n"
"   nop \n"
"   bne 1b \n"
: [delay] "+l"(delay)
);
}

if the "volatile" was not there?