arm-none-eabi-gcc (bleeding-edge-toolchain) 9.1.0

[Freddie Chopin]

I've just created another release of my bleeding-edge-toolchain script, which builds a complete toolchain for ARM microcontrollers. This toolchain uses the most recent versions of all possible components:

• gcc-9.1.0
• newlib-3.1.0
• binutils-2.32
• gdb-8.2.1
• expat-2.2.6
• gmp-6.1.2
• isl-0.21
• libiconv-1.16 (Windows only)
• mpc-1.1.0
• mpfr-4.0.2
• python-2.7.16 (Windows only)
• zlib-1.2.11

The script itself and precompiled packages for 32-bit and 64-bit Windows are available on http://www.freddiechopin.info/ > Download > Software > bleeding-edge-toolchain. You can also find the script on github https://github.com/FreddieChopin/bleeding-edge-toolchain/releases/tag/190503

[ale500]

Did you do any comparisons in terms of compile time, size of executable, performance, just curious.

[Freddie Chopin]

Did you do any comparisons in terms of compile time, size of executable, performance, just curious.

Are you interested in comparison to the "official" GCC toolchain provided by ARM or maybe in comparison to the previous versions of GCC? Generally it seems that GCC 9 is a bit slower than previous versions (a constant trend in all previous major versions of GCC...), but this is not a significant difference - somewhere around 5%. Comparing size or performance is a bit tricky, as you need a good "test sample" to come to any useful conclusion...

[ralphrmartin]

How can anything using python 2 be considered bleeding edge?

[krho]

How can anything using python 2 be considered bleeding edge?

My thoughts exactly. Python 2 will be dead early next year.

[Freddie Chopin]

Sure, you are right, but... (; There is always a "but" (;

Python is required for GDB to enable Python scripting, needed (for example) for gcc's libstdc++v3 pretty-printers (so you could - during the debugging session - see the contents of things like std::vector instead of useless details of the implementation). On Linux there is no problem, as GDB will use Python version installed in the system (either 2 or 3), however on Windows things are not so nice - last time I tried, GDB for Windows did _NOT_ work with Python 3, internally it used functions which are available in Python 2 only. Maybe recently it changed and you can use Python 3 with Windows too, but maybe not and Windows users are still stuck with Python 2 for GDB. Will probably try again with Python 3 sometime, but this is low priority for me - I don't use Windows and probably 99,666% users of microcontrollers don't need Python scripting in GDB anyway.

Please however note that Python 2 used by this toolchain affects only Windows users and only GDB - it has nothing to do with the process of compilation and linking, just the debugging and even then, only when you need Python scripting. Therefore the toolchain is as-bleeding-edge-as-possible (;

[Nominal Animal]

As a side note to those using gdb for debugging:  A few years ago I posted at stackoverflow a Python pretty-printer example, for some basic types; with examples on how to use them.

[janoc]

Actually, python scripting in GDB is pretty nice and useful - e.g. I am using a script that loads an SVD file and allows to inspect directly the various peripheral registers. Pretty useful if you don't want to fire up/configure an entire IDE like Eclipse just to quickly check a value.

But they really should update it to work with Python3.

This is the set of scripts if someone is interested:
https://github.com/bnahill/PyCortexMDebug

[Harjit]

Freddy - thanks for ARM version of the toolchain!

The package has many components - list below. If all I want is the equivalent to what ARM releases, which of these do I need? I was think it should be gcc, binutils, gdb and python. But then I remember that ARM puts gdb into same bin directory as the compiler. I guess I'm asking how do I go from your package to a similar structure to what ARM provides?

•gcc-9.1.0
•newlib-3.1.0
•binutils-2.32
•gdb-8.2.1
•expat-2.2.6
•gmp-6.1.2
•isl-0.21
•libiconv-1.16 (Windows only)
•mpc-1.1.0
•mpfr-4.0.2
•python-2.7.16 (Windows only)
•zlib-1.2.11

[Harjit]

Freddy, it looks like you generate essentially the same structure as the ARM toolchain.

It seems the libgcc.a in your build are around 19.5MB but the ARM releases are 1.5MB. Is this because you are building with symbols or?

[Freddie Chopin]

Freddy, it looks like you generate essentially the same structure as the ARM toolchain.

Yes, exactly. I'm just using the most recent versions where possible and I've tweaked some options a bit to suit my own preferences (; My script is strongly influenced by ARM's script, so both ARM's toolchain and bleeding-edge-toolchain provide basically the same functionalities in basically the same way. Or in other words, bleeding-edge-toolchain is a drop-in replacement for ARM's toolchain.

It seems the libgcc.a in your build are around 19.5MB but the ARM releases are 1.5MB. Is this because you are building with symbols or?

The size difference is because I did not strip the libraries, so they include symbols. This way you can debug the internals of the libraries (; If you want them to be smaller, the stripping can be done anytime you want.

[Harjit]

Thanks for the information.

I looked around but I couldn't find the NewLib compile/link options. Can you point me to them? I want to see how things are configured relative to my project.

[Freddie Chopin]

I looked around but I couldn't find the NewLib compile/link options. Can you point me to them? I want to see how things are configured relative to my project.

Here are the relevant parts of the script:
1. common flags - https://github.com/FreddieChopin/bleeding-edge-toolchain/blob/master/build-bleeding-edge-toolchain.sh#L394
2. additional nano-specific flags - https://github.com/FreddieChopin/bleeding-edge-toolchain/blob/master/build-bleeding-edge-toolchain.sh#L726
3. additional flags specific to non-nano variant of the libs - https://github.com/FreddieChopin/bleeding-edge-toolchain/blob/master/build-bleeding-edge-toolchain.sh#L742

So the nano-variant uses 1 + 2 and the "standard" variant uses 1 + 3.

[SiliconWizard]

First note: I built this toolchain on Linux for both native and Win64 using the script. It went OK for the most part, although there were a couple hiccups and I had to relaunch the script. It takes a pretty long time (over 3 hours).
One thing I noted: there are a lot of apparently useless files copied to the Win64 install, some of them being strictly Linux executable files, others being clones of whole directories (such as the lib directory inside arm-none-eabi\lib...) which makes the whole thing huge (over 4GB!!). Could you look into that? It seems like 1/ some unnecessary files are copied from the Linux build and 2/ newlib gets built twice instead of just once during the Win64 build for some reason...

After some manual cleanup, it went down to about 2GB which is close to the Linux install and still seems pretty big compared to the ARM official builds - although admittedly with older GCC versions. And I did have to strip all executables manually.

I'm not sure what went wrong though, because the builds you're distributing don't exhibit any of these problems (and the binaries seem all stripped), so something may have gone wrong with the script execution on my Linux machine... and it may be due to the hiccups I got while the script was executing. (On Arch Linux for information.)

I also tried to modify your script to build the toolchain natively on Windows (MSYS2), but ended up getting an intractable error with file paths, so I gave up. I managed to build older versions of GCC on MSYS2 in the past though, so it should be doable.

Anyway: I tried this new toolchain on a couple projects (STM32L4) of mine. I didn't notice any significant difference in compile times (with -O3) between gcc 9.1.0 and 8.2.0. There's probably some as quite a few optimizations have been added, but it's almost negligible, at least on my source code. The resulting code size is slightly greater than with 8.2.0 (+5-10% depending on the project) which again probably all comes down to the optimizations being done (again with -O3).

For a summary of changes for GCC 9: https://gcc.gnu.org/gcc-9/changes.html

Amongst many changes, one thing to note is that GCC diagnostics are now a bit easier to read.
Another one is that the D language is now an integral part of GCC. Not sure it can even build for ARM targets (didn't try) but I kinda doubt it.

As a side note, I tried adding support for ADA, but it chokes on libraries. I'm not sure what needs to be done to build ADA support in GCC for ARM - adacore does it...

[Harjit]

Freddy, thank you for the links.

I wanted to understand the impact of the options you have in common flags - https://github.com/FreddieChopin/bleeding-edge-toolchain/blob/master/build-bleeding-edge-toolchain.sh#L394

and looking at NewLib-3.1.0/NewLib/README, I couldn't find: --enable-newlib-retargetable-locking

Not sure if it is no longer supported or what else is going on.

[nctnico]

Just a side question: how nano are the nano libraries? I tried to use newlib (from arm GCC at Launchpad) but this got quite big and required stubs for some IO routines.

[Freddie Chopin]

and looking at NewLib-3.1.0/NewLib/README, I couldn't find: --enable-newlib-retargetable-locking

Not sure if it is no longer supported or what else is going on.

I guess that the README file is not updated. For the complete list of supported flags you could try this:

Code: [Select]

$ pwd
/home/freddie/bleeding-edge-toolchain/sources/newlib-3.1.0/newlib
$ ./configure --help
...
  --enable-newlib-retargetable-locking    Allow locking routines to be retargeted at link time
...
This flag basically allows you to have something like this - https://github.com/DISTORTEC/distortos/blob/master/source/newlib/locking.cpp - in your project, so that things from newlib behave properly when used from multiple threads. This mostly affects stdio, so things like `FILE*`. If you are not using an RTOS then this flag has no benefit for your project.

--

Just a side question: how nano are the nano libraries? I tried to use newlib (from arm GCC at Launchpad) but this got quite big and required stubs for some IO routines.

To be honest I never tried the nano libraries, so I have no opinion on them other than the set of options used is a bit "too reduced" for my taste, but this probably depends on what your projects are doing and what you need. I/O stubs are required in all cases - both for nano and non-nano variants, but probably only for functions which operate on stdin, stdout and stderr, so maybe if you use fprintf() or sprintf() instead of printf(), the I/O stubs won't be needed (or not all of them)?

--

@SiliconWizard - I saw your post above, but I have to leave my PC now, so I'll reply to your message later (; Sorry for the delay!

[Freddie Chopin]

One thing I noted: there are a lot of apparently useless files copied to the Win64 install, some of them being strictly Linux executable files, others being clones of whole directories (such as the lib directory inside arm-none-eabi\lib...) which makes the whole thing huge (over 4GB!!). Could you look into that? It seems like 1/ some unnecessary files are copied from the Linux build and 2/ newlib gets built twice instead of just once during the Win64 build for some reason...

The copying of the libraries is intentional - there is no point in building all the target libraries again. However copying the executables is definitely not OK. Is it possible for you to upload the package(s) which was built on your PC somewhere? Maybe it would even be possible to run the script again, but saving all of the output to log files for closer inspection?

After some manual cleanup, it went down to about 2GB which is close to the Linux install and still seems pretty big compared to the ARM official builds - although admittedly with older GCC versions. And I did have to strip all executables manually.

I'm not sure what went wrong though, because the builds you're distributing don't exhibit any of these problems (and the binaries seem all stripped), so something may have gone wrong with the script execution on my Linux machine... and it may be due to the hiccups I got while the script was executing. (On Arch Linux for information.)

This is indeed strange - in here the windows toolchain - after extracting it - is 2.3 GB. Comparing with previous gcc version is not an indication of a problem in the script (at least I think so [; ). In here the packages for GCC 8 were MUCH smaller than the ones for GCC 9 - compressed for linux is 114 MB vs 215 MB, extracted - 1.8 GB vs 2.2 GB.  I've looked into that a few days ago and the main difference - apart from slightly larger libraries (libstdc++ is 28 MB vs 19 MB and there are 20 version of this lib)  - seems to be the fact that now the documentation folder includes doxygen docs for libstdc++ (~100 MB of HTML).

A log of script execution would be very helpful - just make sure to log both stdout and stderr (either to single file or to two separate files, doesn't matter).

I also tried to modify your script to build the toolchain natively on Windows (MSYS2), but ended up getting an intractable error with file paths, so I gave up. I managed to build older versions of GCC on MSYS2 in the past though, so it should be doable.

This seems like a tough job to do, especially when you can run Linux in a virtual machine... Anyway - to work "natively" on Windows/MSYS2 I guess the script would require a massive rework...

Anyway: I tried this new toolchain on a couple projects (STM32L4) of mine. I didn't notice any significant difference in compile times (with -O3) between gcc 9.1.0 and 8.2.0. There's probably some as quite a few optimizations have been added, but it's almost negligible, at least on my source code. The resulting code size is slightly greater than with 8.2.0 (+5-10% depending on the project) which again probably all comes down to the optimizations being done (again with -O3).

The difference in time is negligible - around 5% usually - and for size comparison you could try -O2 or -Os, as in -O3 a lot of advanced optimizations are going on, with a lot of tweakable parameters, so each version may decide to unroll the loops of inline functions slightly differently. -O2 is a bit more consistent here. Unless you really are after max performance, then -O3 is a good choice (however it's not always the fastest...).

As a side note, I tried adding support for ADA, but it chokes on libraries. I'm not sure what needs to be done to build ADA support in GCC for ARM - adacore does it...

You could try to get scripts adacore uses (maybe they publish it?) and compare then.

[SiliconWizard]

@Freddie Chopin: thanks for your detailed answers.

Regarding the copying of the libraries: yes I had figured that out and it does make the Windows builds much faster indeed.

I took the time to investigate the issue I encountered a bit further and I think I nailed it. Logged the stdout and stderr like so:

Code: [Select]

./build-bleeding-edge-toolchain.sh > stdout.log 2> stderr.log
Turns out the first time I launched your script, I actually used the 'dot' command for some reason instead of running it directly:

Code: [Select]

. ./build-bleeding-edge-toolchain.sh
The main difference is that it would run the script in the current bash session instead of starting a new one. Thanks to the logging, I figured what went wrong with this: the script was failing during the final steps of each build, and running it with the 'dot' command would make it exit the bash session upon an error and thus terminate the script, which doesn't happen if you run the script directly.

The "offending" parts I think are the following:
* For the native build: it's the "strip" phase (line 757):

Code: [Select]

find ${installNative} -type f -executable -exec strip {} \; || trueThe "strip" command fails for some of the files which are not "strippable" although being executables (eg: shell scripts), and it makes the script exit the bash session.
* For the Windows build: it's the "cp" phase (line 929):

Code: [Select]

cp /usr/${triplet}/bin/${dll} ${installFolder}/bin/ || trueSome DLL's listed int the ${dll} variable can't be found during the copy (at least on my Linux machine) and thus this makes the script exit the bash session.

Those two sets of "errors" don't matter if you run the script directly (and not via the 'dot' command), because it won't end the script's execution then, so they are harmless then.

Once I figured that, I then ran the script directly and it worked correctly.

Regarding the overall size of the GCC distribution, it looks like it's mainly due to the very inflated size of the 'arm-none-eabi/lib/thumb' subdirectory (I think it's due to the fact that libraries there all embed debug info whereas they don't in the official ARM toolchain distribution: to be checked).

Regarding my attempt to build it on MSYS2, it was mainly for the challenge, but I gave up pretty fast. I have two Linux machines on which I can build this.

As to the support of ADA, I don't have the time to investigate that further. The official repos are there: https://github.com/AdaCore

Finally, it all inspired me to build a native GCC 9.1.0 for Linux and Win64. I compared optimizations and overall perfomance with code profiling on a few projects. At first sight, it looks like the performance I get with GCC 9.1.0 compared to 8.2.0 (native) with typically -O3 on Core i7's is rather consistantly slightly worse (but like +1-2% execution time or something). I didn't get to test this further so far, nor do that with arm-none-eabi-gcc on Cortex targets, but I would have expected better rather than worse... of course I may have to investigate optimization options to get a better idea.

[Freddie Chopin]

@SiliconWizard, glad that you found the reason for the failure you experienced. Do you think there is a way to check for such incorrect use in the script and error-our early, with a friendly message telling what went wrong and how the script should be used?

(I think it's due to the fact that libraries there all embed debug info whereas they don't in the official ARM toolchain distribution: to be checked).

The libraries are indeed not stripped (they are stripped in official ARM release), but I guess that stripping them should be as easy as doing sth like

find  . -name '*.a' -print0 | xargs -0 -I{} -t arm-none-eabi-strip {}

(maybe also the same for all *.o files, but I'm not sure it would be worth the effort)

If you don't care about debugging the internals of these libraries, then it's safe to drop the debugging info. I think that having an option for the script which selects whether or not the libs should be stripped (maybe `--keep-debugging-info` or sth like that) would be a good idea for the future (;

[SiliconWizard]

@SiliconWizard, glad that you found the reason for the failure you experienced. Do you think there is a way to check for such incorrect use in the script and error-our early, with a friendly message telling what went wrong and how the script should be used?

I'm not sure there is a way of checking this from within the script itself, but don't waste time on this. You could just add a note in the README or something.

(You could also probably modify your script slightly so that errors from the 'strip' and 'cp' commands that I mentioned would be ignored, not sure how though. I'm still not quite sure why those errors would stop the script's execution when invoked with the dot command and not when invoked directly...)

(I think it's due to the fact that libraries there all embed debug info whereas they don't in the official ARM toolchain distribution: to be checked).

The libraries are indeed not stripped (they are stripped in official ARM release), but I guess that stripping them should be as easy as doing sth like

find  . -name '*.a' -print0 | xargs -0 -I{} -t arm-none-eabi-strip {}

(maybe also the same for all *.o files, but I'm not sure it would be worth the effort)

If you don't care about debugging the internals of these libraries, then it's safe to drop the debugging info. I think that having an option for the script which selects whether or not the libs should be stripped (maybe `--keep-debugging-info` or sth like that) would be a good idea for the future (;

Well adding that option could be nice. Some people may find having debug info in the libraries useful, but I guess many don't and stripping them would save almost 1GB or so!

[nctnico]

Why would you want to save 1GB nowadays? Hard drive space is cheap.

[SiliconWizard]

Just a quick follow-up: adding support for ADA is just a matter of adding the '--disable-libada' option to configure gcc (and obviously adding 'ada' to the list of enabled languages).

libada is basically the ADA runtime and can't be built with newlib (which was the cause of the build issues) for again pretty obvious reasons for embedded targets.

You can then check out: https://github.com/AdaCore/Ada_Drivers_Library for example projects, drivers and runtimes.

Doesn't look completely obvious to use if you're not using AdaCore build tools, but seems doable nevertheless. You can pass the '--RTS=' option to arm-none-eabi-gcc to point to the right runtime for your target. Now I have to figure out how to build a runtime for a specific target...

[Freddie Chopin]

Now I have to figure out how to build a runtime for a specific target...

You may find some info in this blog (all recent articles are about Ada on a STM32), but it is in Polish, so your only option is to google-translate it - https://ucgosu.pl/2019/04/generowanie-plikow-runtime-ady-dla-mikrokontrolerow/