Author Topic: [solved] Linux: maximum bootable kernel size in PPC/32bit  (Read 2362 times)

0 Members and 1 Guest are viewing this topic.

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
[solved] Linux: maximum bootable kernel size in PPC/32bit
« on: September 18, 2021, 12:27:20 am »
This seems like a pretty straightforward question, but extensive experiments from kernel 4.11 to kernel 5.7.19 haven't shown a really clear, up-to-date answer.

k4.11 .. k5.7.19: when the kernel size is bigger than 8 MB, the final kernel doesn't boot but rather it freezes.

Why?  :-//

I'm having one hell of a time compiling monolithic kernels and keeping then under 8M without leaving out some core networking and file-system features I need; for several reasons I cannot use modules.

Anyway, it's not clear to me why a 32 bit processor would not be able to boot a kernel larger than 8 MB  :o
« Last Edit: September 27, 2021, 03:52:00 pm by DiTBho »
 

Offline PKTKS

  • Super Contributor
  • ***
  • Posts: 1314
  • Country: br
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #1 on: September 18, 2021, 11:41:22 am »
(.))
Anyway, it's not clear to me why a 32 bit processor would not be able to boot a kernel larger than 8 MB  :o

This is far from a correct postulation.
I can boot a raw 486 or 586 legacy board with 32 MEGA FPM/EDO memory.

You need as well to post a whole lot more details about what you have done on kernel to have issues.  Generally a minimal kernel excludes everything besides bare bones memory manager and basic kernel threads.

Everything else goes via INITRD and RAMDISK_SIZE like:
Code: [Select]
append="initrd=isoroot.gz  ramdisk_size=xxxxxxx  root=/dev/ram0"

Of course the base bare bones devices are also required in your isoroot image.
and grub is a PIG with such options.. 

The whole modern thing (udev/systemd/grub) will likely NOT WORK in minimal setups
a lot of tinkering and squashfs is usually done  unlike  bare isorootfs

This is not trivial and can not be based on these stock "modern" shit kernels.

Paul
« Last Edit: September 18, 2021, 11:43:13 am by PKTKS »
 

Offline thinkfat

  • Supporter
  • ****
  • Posts: 1752
  • Country: de
    • Matthias' Hackerstübchen
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #2 on: September 18, 2021, 12:23:55 pm »
This is typically a problem of relocations not being resolved properly during linking the kernel. Relative jumps end up in nirvana instead of the right target address. Watch out for linker warnings during the final stage of the kernel build
Everybody likes gadgets. Until they try to make them.
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #3 on: September 18, 2021, 01:31:46 pm »
Everything else goes via INITRD and RAMDISK_SIZE like:

There is no initrd or similars, the kernel mounts an hard-drive.

If the kernel size is > 8MB, it doesn't boot, it doesn't show anything, it simply freezes without any clue.
If the kernel size is <= 8Mbyte, it boots correctly.

 
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #4 on: September 18, 2021, 01:35:02 pm »
This is typically a problem of relocations not being resolved properly during linking the kernel. Relative jumps end up in nirvana instead of the right target address. Watch out for linker warnings during the final stage of the kernel build

Yup, this was/is a problem with ARM-classic, and it's also a problem with HPPA2 due to relative jumps. But this is something it shouldn't happen with PowerPC.

Anyway, there is also a second problem: the boot-wrapper (bZimage) cannot copy the kernel to - say 0x00010000 - it must copy the kernel to 0x00000000, otherwise it won't boot.

This way I cannot easily attach a serial debugger, unfortunately I don't have a jtag debugger and KGDB simply dies
 

Offline PKTKS

  • Super Contributor
  • ***
  • Posts: 1314
  • Country: br
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #5 on: September 18, 2021, 01:41:22 pm »
Everything else goes via INITRD and RAMDISK_SIZE like:

There is no initrd or similars, the kernel mounts an hard-drive.

If the kernel size is > 8MB, it doesn't boot, it doesn't show anything, it simply freezes without any clue.
If the kernel size is <= 8Mbyte, it boots correctly.

Hard to make sense..
You strip something from the core and it boots...
So you have some shit modules being trashing the whole chain..

Stripping more and more eventually will lead you to INITRD.

If  you can boot the minimal one why are you bothering with the larger?
usually it is the other way around..

we bother to increase the minimal build with as much as possible..
not the other way around

Paul
 

Offline thinkfat

  • Supporter
  • ****
  • Posts: 1752
  • Country: de
    • Matthias' Hackerstübchen
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #6 on: September 18, 2021, 01:47:59 pm »
I've seen relocations fail also on Arm v8, when the rootfs was linked into the kernel. But then the kernel completely failed to link.
Everybody likes gadgets. Until they try to make them.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7267
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #7 on: September 18, 2021, 02:01:29 pm »
If the kernel size is > 8MB, it doesn't boot, it doesn't show anything, it simply freezes without any clue.
If the kernel size is <= 8Mbyte, it boots correctly.

I'd be focusing on the bootloader. And ignoring PKTKS who can't even tell you're not dealing with an x86.

An interesting experiment might be padding a bootable image to >8M.
 
The following users thanked this post: DiTBho

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #8 on: September 18, 2021, 02:48:40 pm »
If  you can boot the minimal one why are you bothering with the larger?

Because I need several features built-in and I cannot compile them as modules.
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #9 on: September 18, 2021, 03:18:33 pm »
I'd be focusing on the bootloader

u-Boot-2017 ;D

The booting flow is something like:

u-Boot
-> tftpload uImage (uncompressed)
-> park the uImage at 0x0010.0000 0x0001.0000
-> bootm, unroll into binary image at 0x0080.0000 (this exposes the boot-wrapper and the elf header)
-> check the elf header
-> jump into the boot-wrapper
Kernel
-> copy the kernel to 0x0000.0000 (8MByte copied from 0x0010.0000+x to 0x0000.0000)
-> prepare and finalize the DTS image (flat device tree)
-> copy DTS to somewhere in RAM in the kernel area
-> invoke kentry(addr(dts), 0, NULL), which jumps into %entry
-> boot or die

u-Boot -> unroll -> boot-wrapper -> kernel boot


I added a very wild debug module, which directly uses the serial0, it's not yet initialized by the kernel, it's re-initialized in polling-mode to avoid interrupts.

Code: [Select]
image loaded from 0x00800000
SP=0x03eb1b80
kernel_size = 7411084 bytes
copying 256 bytes from kernel-image at 0x0080f000 to elfheader
elf_info.loadsize = 0x00700e68
elf_info.memsize  = 0x0074234c
allocating 7611212 bytes for the new kernel
copying ...
from = 0x0081f000
to = 0x00000000
size = 7343720
flush_cache, 32Mbyte flushed
cmdline: uboot bootargs overridden
cmdline=[console=ttyS0,115200 root=/dev/sda2 rootfstype=ext2 rw init=/sbin/init ]
Finalizing device tree... flat tree at 0xf23b80
ft_addr=0xf23b80
my tp1: success
my tp2: success
my tp3: success
invalidate_cache 0x00000000+0x02000000
my tp4: (point of no return)
calling kentry()...

kernelinfo version 5.2.1
kernelinfo compiled by DiTBho@minerva
kernelinfo compiled with gcc version 9.3.0
kernelinfo build ticket #66 PREEMPT Sat Sep 18 2021


maybe there is something wrong with "load_addr"

when load_addr="0x00800000" the kernel boots
when load_addr="0x00900000" the kernel doesn't boot
« Last Edit: September 18, 2021, 04:36:13 pm by DiTBho »
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #10 on: September 18, 2021, 03:29:09 pm »
I've seen relocations fail also on Arm v8, when the rootfs was linked into the kernel. But then the kernel completely failed to link.

Three years ago I saw it with HPPA2 kernels. Since the code density is not good, those kernels are usually 12 Mbytes in size and some of their branch instructions fail at run-time. At some point it got somehow fixed but there were no connection errors or warnings.

I remember something similar with intel PXA and ARM classic, never seen before with PowerPC  :-//
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #11 on: September 18, 2021, 03:43:54 pm »
This is a 7.4 MB kernel in size, compiled with

Code: [Select]
    load_addr="0x00900000"
    exec_addr="0x901b00"

Code: [Select]
#lsprettysize output/kernel.uImage
        7. 4 Mbyte output/kernel.uImage


Code: [Select]
image loaded from 0x00900000
SP=0x03eb1b80
kernel_size = 7411084
copying 256 bytes from kernel-image at 0x0090f000 to elfheader
elf_info.loadsize = 0x00700e68
elf_info.memsize  = 0x0074234c
allocating 7611212 bytes for the new kernel
copying ...
from = 0x0091f000
to = 0x00000000
size = 7343720
flush_cache, 32Mbyte flushed
cmdline: uboot bootargs overridden
cmdline=[console=ttyS0,115200 root=/dev/sda2 rootfstype=ext2 rw init=/sbin/init ]
Finalizing device tree... flat tree at 0x1023b80
ft_addr=0x1023b80
my tp2: success
my tp3: success
invalidate_cache 0x00000000+0x02000000
my tp4: (point of no return)
calling kentry()...

it dies at this point, but without a debugger it's like watching something fall into a black hole  :o


Either way, it smells like something related to "load_addr"
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7267
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #12 on: September 18, 2021, 03:53:08 pm »
-> park the uImage at 0x0010.0000

1M.

Quote
-> bootm, unroll into binary image at 0x0080.0000 (this exposes the boot-wrapper and the elf header)

8M.

Have you considered following sense and documentation and not placing your image where it will get overwritten?

A little light googling also suggests there may well be an 8M limit on kernel size depending on your u-boot and kernel configurations, but I'm not feeling like diving too deep into an embedded platform I don't need.
« Last Edit: September 18, 2021, 03:56:52 pm by Monkeh »
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #13 on: September 18, 2021, 04:04:32 pm »
Have you considered following sense and documentation and not placing your image where it will get overwritten?

There is no documentation, the board worked with the current setup until my colleague needed to update the kernel and to add more features, and for a lot of reasons we cannot use the memory above 0x00x0.0000 without modifying u-boot (again).

I am working just right now to a patch to use the memory at 0x0100.0000, which is currently reserved for an on-board FPGA that shares the ram with the SoC, and to an hardware patch to install up to 256-512MB of ram dedicated to the SoC.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7267
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #14 on: September 18, 2021, 04:06:27 pm »
Have you considered following sense and documentation and not placing your image where it will get overwritten?

There is no documentation

u-boot comes with plenty, including a nice note about not overwriting your tftp loaded image.

Quote
for a lot of reasons we cannot use the memory above 0x00x0.0000 without modifying u-boot (again).

Ah, the joys of trying to support ultra-legacy hardware hacks.
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #15 on: September 18, 2021, 04:08:19 pm »
A little light googling also suggests there may well be an 8M limit on kernel size depending on your u-boot and kernel configurations, but I'm not feeling like diving too deep into an embedded platform I don't need.

Where? What?  :o
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7267
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #16 on: September 18, 2021, 04:12:01 pm »
A little light googling also suggests there may well be an 8M limit on kernel size depending on your u-boot and kernel configurations, but I'm not feeling like diving too deep into an embedded platform I don't need.

Where? What?  :o

Bits and pieces like this: https://lists.denx.de/pipermail/u-boot/2016-July/261087.html

Again, don't feel like diving - no idea what platform, exact u-boot version you have, configuration, non-upstream hacks, yadda yadda, nigh impossible to do more than point vaguely and hope you find the right direction..

Oh, and stuff like this: https://lists.denx.de/pipermail/u-boot/2016-July/261088.html ...
 
The following users thanked this post: DiTBho

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #17 on: September 18, 2021, 04:35:33 pm »
-> park the uImage at 0x0010.0000

1M.

double checked, in my post I typed 0x0010.0000 but it is 0x0001.0000
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7267
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #18 on: September 18, 2021, 04:42:01 pm »
-> park the uImage at 0x0010.0000

1M.

double checked, in my post I typed 0x0010.0000 but it is 0x0001.0000

Okay, and if you're writing to 0x800000 later you're clobbering any image bigger than 0x7F0000.
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #19 on: September 18, 2021, 06:59:32 pm »
Okay, and if you're writing to 0x800000 later you're clobbering any image bigger than 0x7F0000.

Not a problem, the last 64Kbytes are empty in every tested kernel and, just to be sure it's not the problem, the tested kernel was compiled with "space optimization" with a size of 7.5 MB

However, although the size is 7.5MB, even this kernel doesn't boot when load_addr="0x0090.0000", no matter where you tftpload it in ram, in fact, I have just modified u-boot to disable the FPGA so I reassigned its ram up to 0x0400.0000, and tftploading at 0x0100.0000 and 0x0200.0000 doesn't solve anything.

this doesn't work:
tftpload uImage (not compressed) -> 0x0200.0000
bootm from 0x0200.0000, copies the wrapper (7.5MB) into 0x0090.0000 (load_addr)
the wrapper copies the kernel into 0x0000.0000 and invokes kentry
the kernel dies immediately without showing anything

this does work:
tftpload uImage (not compressed) -> 0x0200.0000
bootm from 0x0200.0000, copies the wrapper (7.5MB) into 0x0080.0000 (load_addr)
the wrapper copies the kernel into 0x0000.0000 and invokes kentry
the kernel boots


Here the problem is somehow related to load_addr

I also tried load_addr =
{
          0x0081.0000     8M +  64K  -> it works
          0x0082.0000     8M + 128K -> it works
          0x0084.0000     8M + 256K -> it works
          0x0088.0000     8M + 512K -> it works
          0x0090.0000     8M + 1M -> it doesn't works
}
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #20 on: September 18, 2021, 07:03:03 pm »
This is the relation between the kernel size and link_addr
Code: [Select]
        # Round the size to next higher MB limit
        round_size=$(((strip_size + 0xfffff) & 0xfff00000))

        round_size=0x$(printf "%x" $round_size)
        link_addr=$(printf "%d" $link_address)

and this is where link_addr is involved
Code: [Select]
        text_start="-Ttext $link_address"
 

Offline thinkfat

  • Supporter
  • ****
  • Posts: 1752
  • Country: de
    • Matthias' Hackerstübchen
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #21 on: September 18, 2021, 07:09:53 pm »
Well, this is the point where you will need to spike the startup code with uart output, to see where it runs into the woods.
Everybody likes gadgets. Until they try to make them.
 
The following users thanked this post: DiTBho

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #22 on: September 18, 2021, 07:42:39 pm »
Well, this is the point where you will need to spike the startup code with uart output, to see where it runs into the woods.

Already done, see a few posts above  :o

Unfortunately I am lost after kentry(), which is the point of no return in the booting code because it points and jumps into 0x0000.0000, which is the interrupt vector area in PowerPC, and I cannot add a serial debugger there (kind of gdb-stub) because the kernel is already died at that point; I would need a jtag debugger to see and track something down to the code.

As temporary workaround I am forcing the load address to 0x0088.0000, so I can boot a 8. 4 Mbyte kernel, which is a good "compromise".

Why is life so complex? :-//
 

Offline PKTKS

  • Super Contributor
  • ***
  • Posts: 1314
  • Country: br
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #23 on: September 19, 2021, 08:19:20 am »

I'd be focusing on the bootloader. And ignoring PKTKS who can't even tell you're not dealing with an x86.


I always miss your pedantic insults
They make my days

Paul
 

Offline DiTBho

  • Super Contributor
  • ***
  • Posts: 1057
  • Country: gb
Re: Linux: maximum bootable kernel size in PPC/32bit
« Reply #24 on: September 21, 2021, 11:31:27 am »
Yesterday I completed the hardware hack so I can remap the memory registers of the FPGA to 0x4000.0000 (1G), but still the SoC cannot use the ram from 0x0000.0000 to 0x1000.0000 (256MB), but only from 0x0000.0000 to 0x0800.0000 (128 MB).

Not perfect, but better, and now I can park a tftpimage at 0x0800.0000  :D

Older kernels (2.4.* and 2.6.15) were simpler and smaller than current 5.*  :o :o :o

u-boot dropped support for PPC 4xx in 2017.
Linux dropped support for PPC 4xx with the 5.8.0 kernel.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf