[SOLVED] building a metalbare cross compiler from scratch (custom configured)

[legacy]

hi guys
I need to compile gcc, target=mips-be (host=arm-le, I can native compile under Qemu/Arm)
I am a bit confused by the GNU documentation, the following is my current-build-script

Code: [Select]

VERSION=v4.4.3

export TARGET=mips-idp-elf
export PREFIX=/usr/$TARGET
export PATH=$PATH:$PREFIX/bin

target=build/$TARGET-gcc-$VERSION

echo "building gcc"
rm -fr $target
mkdir -p $target
cd $target
../../src/gcc-$VERSION/configure \
         --target=$TARGET \
         --prefix=$PREFIX \
         --disable-libssp \
         --without-headers \
         --with-float=soft \
         --without-newlib \
         --with-gnu-as \
         --with-gnu-ld\
         --disable-nls \
         --disable-threads \
         --disable-shared \
         --disable-libssp \
         --enable-languages=c
make all
make install

I need to compile

• a crosscompiler
• C language only
• target=mips big endian
• profiled for metalbare (no OS)
• I do not need any newlibc (by Cygnus/RedHat) or glibc(by GNU) or whatever (1)

(1) I do not need any newlibc or glibc or whatever, because i have to use my C runtime (crt0) and my BSP, which already includes the full support for libc


* * * My questions * * *

• 1) is the configuration correct? Do I miss something?
• 2) is there a way to reduce the time needed to complete the build-phase? I have seen the compiler is rebuild 3 times 
• 3) I do not like the "install-path" (see the following), I'd like to have
• 3.1) { cc1, collect2 } installed in "/usr/mips-idp-elf/bin2" instead of "/usr/mips-idp-elf/libexec/gcc/mips-idp-elf/4.4.3/", how to do so ?
• 3.2) I do not need "/usr/mips-idp-elf/lib/*", and I'd like to have it installed in "/usr/mips-idp-elf/lib" instead of "/usr/mips-idp-elf/lib/gcc/mips-idp-elf/4.4.3/", how to do so ?
• 3.3) I'd like to have include-path installed in "/usr/mips-idp-elf/include", how to do so ?

Code: [Select]

# ls /usr/mips-idp-elf/
bin/          include/      info/         lib/          libexec/      man/          mips-idp-elf/ share/       

 # ls /usr/mips-idp-elf/libexec/gcc/mips-idp-elf/4.4.3/
cc1            collect2       install-tools/

 # ls /usr/mips-idp-elf/lib/gcc/mips-idp-elf/4.4.3/
crtbegin.o     crti.o         el/            include-fixed/ libgcc.a       soft-float/   
crtend.o       crtn.o         include/       install-tools/ libgcov.a

thank you in advance 


edit:
I forgot: mips-binutils already compiled and installed in "usr/mips-idp-elf"
no problem with them 

[legacy]

I am studying gnu dot org configure, but … 

[nctnico]

AFAIK compiling GCC from scratch = world of pain
I'd look for a known-good build script or pre-build toolchain and go from there. There are options to make gcc/ld ignore the standard libraries and startup files. I recommend to use a prebuild GCC and use those options for bare metal work.

[andersm]

Whatever reasons you think you have for breaking the toolchain are not valid.

[legacy]

I recommend to use a prebuild GCC and use those options for bare metal work.

unfortunately it's not already existing, and what exists doesn't satisfy my needs, so I have to recompile 
(I am not a GNU fan, so you can understand I am not happy to have to deal with their sources, but I have to)

[legacy]

breaking the toolchain are not valid.

could You answer my specific questions instead?

[legacy]

( specifically host=armv5tejl-linux-gnueabi )

[janoc]

Hello,

I haven't built GCC for mips, but I did compile it for AVR and ARM before. So take my opinion with a grain of salt.

Code: [Select]

VERSION=v4.4.3

export TARGET=mips-idp-elf
export PREFIX=/usr/$TARGET
export PATH=$PATH:$PREFIX/bin

target=build/$TARGET-gcc-$VERSION

echo "building gcc"
rm -fr $target
mkdir -p $target
cd $target
../../src/gcc-$VERSION/configure \
         --target=$TARGET \
         --prefix=$PREFIX \
         --disable-libssp \
         --without-headers \
         --with-float=soft \
         --without-newlib \
         --with-gnu-as \
         --with-gnu-ld\
         --disable-nls \
         --disable-threads \
         --disable-shared \
         --disable-libssp \
         --enable-languages=c
make all
make install


I need to compile

• a crosscompiler
• C language only
• target=mips big endian
• profiled for metalbare (no OS)
• I do not need any newlibc (by Cygnus/RedHat) or glibc(by GNU) or whatever (1)

AFAIK, the config above will build a native compiler (i.e. something that will run on a MIPS machine), not a crosscompiler. You need to add the --host option for that.

You can look here for instructions and cherrypick the relevant bits (they are compiling a toolchain targetting Linux on MIPS, though - different TARGET):
https://www.linux-mips.org/wiki/Toolchains

(1) I do not need any newlibc or glibc or whatever, because i have to use my C runtime (crt0) and my BSP, which already includes the full support for libc

Then just don't compile newlib/glibc.

* * * My questions * * *

• 2) is there a way to reduce the time needed to complete the build-phase? I have seen the compiler is rebuild 3 times 

If I remember right, GCC builds first a bootstrap compiler, then it uses that to compile the gcc version you want to build and finally uses the self-compiled gcc to rebuild itself to ensure correctness. So no, from this point of view there isn't a way to speed it up.

What you can play with is to ensure that flags such as -pipe are set, that will avoid using temporary files between preprocessor and compiler and compiler and assembler, using pipes instead. I wouldn't recommend doing parallel build (make -j X, where X is the number of parallel builds you want) unless the GCC doc says this is OK - the GCC build is very complex and things could go wrong if it is not adapted for parallel compilation.

• 3) I do not like the "install-path" (see the following), I'd like to have
• 3.1) { cc1, collect2 } installed in "/usr/mips-idp-elf/bin2" instead of "/usr/mips-idp-elf/libexec/gcc/mips-idp-elf/4.4.3/", how to do so ?
• 3.2) I do not need "/usr/mips-idp-elf/lib/*", and I'd like to have it installed in "/usr/mips-idp-elf/lib" instead of "/usr/mips-idp-elf/lib/gcc/mips-idp-elf/4.4.3/", how to do so ?
• 3.3) I'd like to have include-path installed in "/usr/mips-idp-elf/include", how to do so ?

No idea how to do this - you may have to hack on the makefiles. I suggest that you ask on the GCC mailing list, because GCC expects these files in certain places and if you change the path it could break stuff.

[legacy]

Code: [Select]

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND     
 2544 root      40   0   52112  35812    272 R 99.2 58.4  18:53.71 genattrtab

also I am not happy with "genattrtab", because it eats more than 180Mbyte of ram during the gcc-buildup 
( I add to add 1Gbyte of ram to Qemu/arm, in order to reduce the swap size, which is handled to the virtual hard drive ---> too slow)

[legacy]

AFAIK, the config above will build a native compiler (i.e. something that will run on a MIPS machine), not a crosscompiler. You need to add the --host option for that.

it works perfectly!
as I wrote above the whole build up is invoked under Qemu/Arm!
so host=arm by default, host=arm, target=mips ---->it's a cross compiler!!!

Then just don't compile newlib/glibc

Yes, but you have to specify if you need support (new lib? I have forced --no-xxx), and I also want to avoid libgloss to be compiles: I do not need it!

If I remember right, GCC builds first a bootstrap compiler, then it uses that to compile the gcc version you want to build and finally uses the self-compiled gcc to rebuild itself to ensure correctness. So no, from this point of view there isn't a way to speed it up.

so --disable-bootstrap isn't working?

No idea how to do this - you may have to hack on the makefiles. I suggest that you ask on the GCC mailing list, because GCC expects these files in certain places and if you change the path it could break stuff.

OK

[legacy]

it works perfectly!

see the log

Code: [Select]

building gcc
checking build system type... armv5tejl-linux-gnueabi
checking host system type... armv5tejl-linux-gnueabi
checking target system type... mips-idp-elf


[Lukas]

Have you tried buildroot? I managed to make it produce a working mips gcc some time ago with little effort.

[legacy]

Have you tried buildroot? I managed to make it produce a working mips gcc some time ago with little effort.

does it solve my specific questions ?

[legacy]

subscribed to  - gcc-digest at gcc dot gnu dot org -

[dgtl]

Why not using crosstool-ng? It helps with a kernel-style menuconfig menu configuring all of the options you might otherwise not think of.

[legacy]

Why not using crosstool-ng? It helps with a kernel-style menuconfig menu configuring all of the options you might otherwise not think of.

because it does not matter with my purpose!

[janoc]

it works perfectly!
as I wrote above the whole build up is invoked under Qemu/Arm!
so host=arm by default, host=arm, target=mips ---->it's a cross compiler!!!

Ah okay, my bad. You are right, of course.

Yes, but you have to specify if you need support (new lib? I have forced --no-xxx), and I also want to avoid libgloss to be compiles: I do not need it!

=> GCC mailing list, that is a fairly non-standard setup.

so --disable-bootstrap isn't working?

You didn't have --disable-bootstrap in your configure options if I see correctly. Furthermore, I wouldn't recommend that, because the compiler you may have on your host may not be able to build the gcc - e.g. some commercial Unix systems come with a really ancient compiler and the bootstrap was designed to overcome this.

If you are building on Linux with a reasonably recent gcc it will probably work, but it is better to be safe than chase weird bugs because some obscure code got optimized wrong.

However, if I understand this page correctly: https://gcc.gnu.org/install/build.html
the bootstrapping is used only for building native compilers, not a crosscompiler. So this could be a moot point.

On the other hand, they say that parallel building is possible, so you may want to try make -j X build. However, if you are building in QEmu, it will likely not help much - there the bottleneck is the emulated CPU and amount of RAM and a parallel build will only make things go even slower.

[janoc]

because it does not matter with my purpose!

Before you yell at people I suggest you actually look at those tools - both buildroot and crosstool-ng. Both are designed to do what you are attempting to do and it is possible that the things you are looking for are configurable there. Worth checking them out, IMHO, instead of trying to reinvent the wheel.

People are trying to help you here, mate.

[legacy]

if you are building in QEmu, it will likely not help much - there the bottleneck is the emulated CPU and amount of RAM

yes, it's slower, no help from the j-flag when you are under QEmu (and see how much memory it eats  )

I am manually trying different options (I have asked the right combination to GNU email list)
the GNU documentation is not so clear to me too, their sources appears too bloated, which is confusing 

I am really afraid I have to hack the Gcc's Makefile to force things
or probably I'd better hack the binary "gcc" in order to force the "collector" to point to the wanted-path 

"cc1" is the C compiler, "gcc" invokes it, so I could, eventually trash "gcc" and build my-own version of the "collector"
I already have my builder (already tested), so a project work-flow will look this way

• { list of file.s } ----> GNU.as ( my FLAGs ) ----> { list of file.o }
• { list of file.c } ----> GNU.cpp (1) --> GNU.cc1 ( my FLAGs ) ----> {{ list of file.o } + { list of file.S }} ( I want to see the assembly level )
• { { list of file.o } + myBSP + myLIBC.o + myCRT0.o } ---> GNU.ld ( my flags + linker script + memory model ) ---> binary (elf+debug symbols)
• binary.elf ---> GNU.objdump (strip) ---> motorola/s19

no doubt that it's a bit crude, but it looks like a practical solution to have everything under control ( including paths )
and it avoids me to waste too much time understanding whatever is GNU-skilled-only 



(1) GCC has a number of phases to its compilation, and it uses different internal commands to do each phase.
C in particular is first preprocessed with cpp, then is compiled into assembly, assembled into machine language, and then linked together.

cc1 is the internal command which takes preprocessed C-language files and converts them to assembly.
It's the actual part that compiles C

Interesting read about Gcc's Architecture

[legacy]

If you are building on Linux with a reasonably recent gcc

ups, I forgot to tell you about that

Code: [Select]

QEmu-arm # gcc -v

Using built-in specs.
COLLECT_GCC=/usr/armv5tel-softfloat-linux-gnueabi/gcc-bin/4.8.4/gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/armv5tel-softfloat-linux-gnueabi/4.8.4/lto-wrapper
Target: armv5tel-softfloat-linux-gnueabi
Configured with: /var/tmp/portage/sys-devel/gcc-4.8.4/work/gcc-4.8.4/configure --host=armv5tel-softfloat-linux-gnueabi --build=armv5tel-softfloat-r
Thread model: posix
gcc version 4.8.4 (Gentoo 4.8.4 p1.6, pie-0.6.1)


[janoc]

Silly question, but do you need to host the compiler on that particular (and ancient - ARM5? Whoa - are you planning to compile on a Sharp Zaurus?) architecture? If you could compile on real hw (e.g. in native Linux or at least in a fast Linux VM like VirtualBox), it would certainly be a lot faster.

Even if you need to host on ARM, a newer system running on something like Raspberry Pi or BananaPi (or one of the many ARM Linux SBCs available now) would be probably much faster than the emulation in Qemu.

BTW, why are you fighting with the paths? The ones that GCC uses by default are semi-standard and other tools may actually depends on them. I wouldn't move things around unless I had a very good reason for that. The scheme GCC uses is there to permit having multiple compiler versions installed side-by-side. Wouldn't a bunch of symbolic links post install solve your problem? (I assume you only want the path changed for convenience). That's probably a lot easier and safer than hacking binaries or the build system where one doesn't understand all the dependencies.

[legacy]

Even if you need to host on ARM, a newer system running on something like Raspberry Pi or BananaPi (or one of the many ARM Linux SBCs available now) would be probably much faster than the emulation in Qemu.

faster than QEmu/ARM on multi core i7 machine  ?

[legacy]

Solved !!!

it works great on QEmu/ARM

two solutions, two branches, mutual exclusive

• hacked "gcc.c", forcing all the paths I need (and I need it this way for my reasons, no time to say why/what/where)
• removed "gcc", swapped it with my "collector", it's a piece of software I have developed, it does the same job of "gcc" (as collector)

edit: lesson learnt from GNU.gcc's configure

• prefix
• gcc_exex_prefix
• C_include_path

in the theory they allow you place the compiler pieces in any location
practically(1) they get embedded within "gcc.c", in the "collector" part

In order to be sure that things get built as I expected:
I had better change and modify "gcc.c" immediately
this is my conclusion!


(1) note these can be used as environment variables to move things around
even on a already built and working compiler, so …
… they might cause troubles to the host's toolchain (it's not clear WHY?) 

[janoc]

Even if you need to host on ARM, a newer system running on something like Raspberry Pi or BananaPi (or one of the many ARM Linux SBCs available now) would be probably much faster than the emulation in Qemu.

faster than QEmu/ARM on multi core i7 machine  ?

Did you benchmark it? If you are hitting swap at 1GB of RAM in the emulator, I think it will be faster if you take a physical, not emulated HW and 2GB of RAM. Qemu is very slow compared to native code. I would be very surprised if even multicore i7 was able to run it at near native speeds.

[legacy]

I do not have any RPI/BananaPI/whatever on hand, and no time to buy/reallocate one of them in the lab
while the customer wants a copy of the whole building system (which should run on PC), so it's not the point

about what I could to to improve my results: I simply assigned less than 256Mbyte of ram to QEmu (my mistake)
and due to the fact that gcc-build-up eats a lot of ram (more than 512Mbyte if you enable "pipe" instead of "temp files")
the kernel was swapping on /dev/sda1, which was a bottleneck, and assigning 1Gbyte of ram to QEmu improved results