Warning to users of xPack RISC-V GCC distribution

[HwAoRrDk]

This is a cautionary note to anyone who is using the xPack riscv-none-elf-gcc distribution to compile code for RV32E processors that support the 'C' compact instruction extension. For example, CH32V003 series.

It seems at some point in the last few months, for some reason the maintainer has decided to reduce the selection of architecture/ABI combinations that GCC's built-in libraries (i.e. libgcc, libc, etc.) are pre-compiled for. Among those, rv32ec/ilp32e is no longer present in the latest releases.

What this means is that any code you compile and link specifying -mabi=ilp32e and -march=rv32ec will be silently 'downgraded' to instead use the rv32e/ilp32e libraries. This is because, when it doesn't have the specified arch/ABI pairing, GCC picks the nearest baseline to what you specify - in this case, rv32e instead of rv32ec. As the non-'c' pre-compiled libraries do not, as the name suggests, make use of any compact 'C'-extension instructions, the size of their code is larger, and therefore your compiled binaries will suddenly become larger, even with no other changes.

The last releases that feature rv32ec libraries are:

• v14.2.0-1
• v13.3.0-1
• v12.4.0-1
• v11.5.0-1

Anything newer (e.g. v13.3.0-2) no longer features rv32ec libraries. I suggest, at present, if you value or critically rely upon smaller code size, to not upgrade past any of the above listed versions. And if you've inadvertently upgraded and didn't realise there was a detrimental change, then you might want to downgrade.

(Note this doesn't mean any of your code that is compiled won't be using compact instructions. This only applies to any pre-compiled library functions that GCC links into your code. For example, multiplication or division routines, save/restore stubs, standard C-library functions, etc.)

[HwAoRrDk]

To give a concrete example:

I compiled an existing CH32V003 codebase of mine with both 13.3.0-1 and 13.3.0-2 and the size of the output binary increased from 13,136 bytes to 13,264 bytes! And note that this is a codebase that only makes minor use of standard C library functions. According to the map file, only the following library functions are being linked - save/restore stubs, some multiplication and division routines, and strcat, memset, and memcpy:

Code: [Select]

C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/rv32e/ilp32e\libgcc.a(save-restore.o)
                              obj\Release\flash.o (__riscv_save_2)
C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/rv32e/ilp32e\libgcc.a(muldi3.o)
                              obj\Release\main.o (__mulsi3)
C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/rv32e/ilp32e\libgcc.a(div.o)
                              obj\Release\uart.o (__divsi3)
C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/../../../../riscv-none-elf/lib/rv32e/ilp32e\libc_nano.a(libc_a-strcat.o)
                              obj\Release\main.o (strcat)
C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/../../../../riscv-none-elf/lib/rv32e/ilp32e\libc_nano.a(libc_a-memset.o)
                              obj\Release\main.o (memset)
C:/[...]/xpack-riscv-none-elf-gcc-13.3.0-2/bin/../lib/gcc/riscv-none-elf/13.3.0/../../../../riscv-none-elf/lib/rv32e/ilp32e\libc_nano.a(libc_a-memcpy-asm.o)
                              obj\Release\i2c.o (memcpy)

To pick out a specific example from the above, __divsi3 was formerly 126 bytes, but afterwards 180.

I could imagine that if one has a codebase that makes much greater usage of stdlib functions, the size difference could be much greater. Maybe in a particularly egregious case you all of a sudden go from just squeezing into the available flash, to no longer fitting at all! And you have no clue why until you go digging in the map file - assuming you are outputting one at all...

[brucehoult]

Have you pointed this out to Liviu?  I'm sure he can be easily persuaded to include variations that people actually use, which is clearly RV32EC for all the very many people out there using the very popular CH32V003.

Or, you can just build binutils/gcc/newlib yourself. Takes about 8 minutes on my laptop for current HEAD (or GCC 14 ... no noticeable difference). And you get to choose whatever options you want, so I find it well worthwhile.

[HwAoRrDk]

Have you pointed this out to Liviu?  I'm sure he can be easily persuaded to include variations that people actually use, which is clearly RV32EC for all the very many people out there using his build.

Not yet - I will do. Not really sure whether to file a 'bug' in a GitHub issue, or perhaps just ask why first. He must've had a reason.

Or, you can just build binutils/gcc/newlib yourself. Takes about 8 minutes on my laptop for current HEAD (or GCC 14 ... no noticeable difference). And you get to choose whatever options you want, so I find it well worthwhile.

I'm using xPack GCC distro for Windows precisely because I don't want to mess around with compiling things myself. Trying to compile *nix-y open-source stuff on Windows is usually a headache.

[SiliconWizard]

Or, you can just build binutils/gcc/newlib yourself. Takes about 8 minutes on my laptop for current HEAD (or GCC 14 ... no noticeable difference). And you get to choose whatever options you want, so I find it well worthwhile.

Same here. But not everyone can do this (and/or feels confident enough to use this instead of a ready-made toolchain). In particular, not all ready-made toolchains do properly support newlib, plus newlib-nano, plus the medany memory model. They also often lag in terms of newer gcc AND binutils releases.

But if the user uses Windows, that's another problem.
I used to build gcc toolchains for Windows... but on Linux. Building it directly on Windows (using for instance MSYS2) used to work but it stopped being an option a good while ago. If you have Windows 10 or 11, I guess you could do this with WSL, but you'll still need to grab a mingw toolchain to be able to build Windows executables on Linux with all required dependencies. That's something that takes a while to figure all out.

[brucehoult]

He must've had a reason.

The obvious reason would be that there are 9999999 possible combinations of extensions, and each combination takes time to build and space in the download / on the disk, so you don't want to do combinations that no one uses.

But rv32ec for CH32V003 and rv32imac_zicsr_zifencei_zba_zbb_zbkb_zbc_zbs_zcb_zcmp for RP2350 are two that there clearly is going to be demand for!

I've been thinking about making a post for this ... I don't know if people are familiar with Zcmp. I was involved with the committee designing it. I'm not entirely convinced that its a good use of silicon compared to alternatives -- and it's DEFINITELY only for simple in-order CPUs (like original 1985 Arm...) -- but as Luke implemented it in Hazard3 it would be silly not to use it as it does save quite a lot of code size.

Example:

Code: [Select]

void move(char from, char to);

void hanoi(char from, char to, char spare, int n) {
    if (n != 0) {
        hanoi(from, spare, to, n-1);
        move(from, to);
        hanoi(spare, to, from, n-1);
    }
}

With Zcmp, 34 bytes of code:

Code: [Select]

hanoi:
        cm.push {ra, s0-s3}, -32
        cm.mvsa01       s0,s3
        mv      s2,a2
        mv      s1,a3
.L3:
        beq     s1,zero,.L1
        addi    s1,s1,-1
        cm.mva01s       s0,s2
        mv      a3,s1
        mv      a2,s3
        call    hanoi
        cm.mva01s       s0,s3
        call    move
        mv      a5,s0
        mv      s0,s2
        mv      s2,a5
        j       .L3
.L1:
        cm.popret       {ra, s0-s3}, 32

Without Zcmp, 62 bytes of code:

Code: [Select]

hanoi:
        addi    sp,sp,-32
        sw      s0,24(sp)
        sw      s1,20(sp)
        sw      s2,16(sp)
        sw      s3,12(sp)
        sw      ra,28(sp)
        mv      s0,a0
        mv      s3,a1
        mv      s2,a2
        mv      s1,a3
.L3:
        beq     s1,zero,.L1
        addi    s1,s1,-1
        mv      a1,s2
        mv      a0,s0
        mv      a3,s1
        mv      a2,s3
        call    hanoi
        mv      a0,s0
        mv      a1,s3
        call    move
        mv      a5,s0
        mv      s0,s2
        mv      s2,a5
        j       .L3
.L1:
        lw      ra,28(sp)
        lw      s0,24(sp)
        lw      s1,20(sp)
        lw      s2,16(sp)
        lw      s3,12(sp)
        addi    sp,sp,32
        jr      ra


[brucehoult]

But if the user uses Windows, that's another problem.
I used to build gcc toolchains for Windows... but on Linux. Building it directly on Windows (using for instance MSYS2) used to work but it stopped being an option a good while ago. If you have Windows 10 or 11, I guess you could do this with WSL, but you'll still need to grab a mingw toolchain to be able to build Windows executables on Linux with all required dependencies. That's something that takes a while to figure all out.

Stupid question, but if you have WSL then why do you need to build a Windows executable?

Even if you're a Windows user, you can just type in PowerShell wsl riscv64-unknown-elf-gcc -O hello.c -o hello or whatever and it will run the Linux toolchain on the hello.c source file in your current Windows directory and leave the RISC-V binary there too.

Starting in February, when I bought a Lenovo i9-13900HX laptop at runout prices (the 14900 version was already out), my main work machine was Windows for the first time in my life, replacing the 32 core 2990WX Threadripper 20+ kg water cooled tower I built in 2019.

The laptop is a little bit faster than the Threadripper for things that keep all the cores busy all the time e.g. Linux kernel or LLVM builds, and a lot faster for things that use a small number of threads or are bursty e.g. GNU builds such as gcc. Plus at 2.5 kg it's 2% lighter than my previous work laptop, a 2011 i7 MacBook Pro. So I can conveniently travel the world with it, use it in cafes etc.

ANYWAY, I basically don't use Windows on it ... just Chrome. And WSL. Most of the time at home the lid is closed and I ssh in from a Mac Mini. I give 22 GB RAM to WSL and 10 GB to Windows 11, which is the least I can give Windows without it becoming completely unusable.

Or at least that's what I was doing until early October, when I got pissed off that Windows doesn't honour taskset done from within WSL. So then I nuked Windows entirely and now the machine has only Ubuntu 24.04. Works great. The only downside I've found is that the battery life with light use (web browsing, editing code, posting messages like this one) is about 5 hours under Linux vs 6 hours under Windows. NBD. It's almost always plugged in, and if I do go to a cafe or something and there is no socket nearby (or I CBF) it's probably maximum 2-3 hours anyway.

But, other than taskset not working, and no direct access to USB devices from WSL (things like RNDIS for the Milk-V Duo work fine, but some flashing tools need to be run in Windows), and 10 GB RAM lost to useless bloat, I was very happy with WSL.

[jnk0le]

What this means is that any code you compile and link specifying -mabi=ilp32e and -march=rv32ec will be silently 'downgraded' to instead use the rv32e/ilp32e libraries.

Still better than "downgrading" to RV32GC it was doing before.

[HwAoRrDk]

It's been suggested to me that it might be possible in the meantime to use rv32eac with the newer xPack versions that lack rv32ec, because so long as nothing from stdatomic.h is used, then no Atomic 'A' extension instructions will end up in the binary.

I'm not convinced whether that would hold true. Any thoughts?

[rhodges]

I'm not convinced whether that would hold true. Any thoughts?

It sounds reasonable. You can disassemble the output and look for any instructions starting with "amo" and lr and sc.

[ejeffrey]

I'm not sure I would want to count on that.  Even if it is true today, I think it's hard to guarantee that a future compiler or library version or changed option might lead to the generation of atomic instructions of they are allowed by the abi. 

I don't know of any examples in C, but I think C++ compilers generally use atomics to guarantee single initialization of static variables with non-trvial constructors.

[brucehoult]

I'm not sure I would want to count on that.  Even if it is true today

It was specified "in the meantime".

The solution obviously is to ask Liviu to include RV32EC in his build, which I'm sure he will do, since it is a combo used on shipping and popular CPUs. He just won't want to have dozens of library versions that no one uses.

I don't know how often he re-does them, but that's likely to be anything from days to maybe a couple of months at most.