Author Topic: ELF to binary for boot loader  (Read 7030 times)

0 Members and 1 Guest are viewing this topic.

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
ELF to binary for boot loader
« on: July 29, 2021, 08:46:00 pm »
There is a lot of stuff online for ELF to BIN etc. For example arm-none-eabi-objcopy can generate the binary.

But not much on how this actually works.

Let's say you are programming a 32F4 within Cube IDE, using STLINK V2/V3. Does the IDE just feed a binary block to the debugger, to program the FLASH with? The debugger has very little storage it in. It must be just programming the stuff straight in, 0x08000000 onwards.

But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening. So whatever is managing this process must be generating multiple binary blocks and programming these individually.

Also, AIUI, for single stepping, you need to have a symbol table "somewhere" and it can't be in the CPU FLASH because a 1MB binary could be several MB with symbols.

So the stuff loaded into the CPU must be just the binary block comprising of the sections 'text' and 'data'.

IOW, same as what you would need to generate for a boot loader which just takes a binary block and programs the CPU FLASH with it.

I am trying to work out how to generate a single binary block. The program will never have discontinuous text or data sections, and AFAIK data always follows text.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 4427
  • Country: dk
Re: ELF to binary for boot loader
« Reply #1 on: July 29, 2021, 09:00:40 pm »
There is a lot of stuff online for ELF to BIN etc. For example arm-none-eabi-objcopy can generate the binary.

But not much on how this actually works.

Let's say you are programming a 32F4 within Cube IDE, using STLINK V2/V3. Does the IDE just feed a binary block to the debugger, to program the FLASH with? The debugger has very little storage it in. It must be just programming the stuff straight in, 0x08000000 onwards.

But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening. So whatever is managing this process must be generating multiple binary blocks and programming these individually.

Also, AIUI, for single stepping, you need to have a symbol table "somewhere" and it can't be in the CPU FLASH because a 1MB binary could be several MB with symbols.

So the stuff loaded into the CPU must be just the binary block comprising of the sections 'text' and 'data'.

IOW, same as what you would need to generate for a boot loader which just takes a binary block and programs the CPU FLASH with it.

I am trying to work out how to generate a single binary block. The program will never have discontinuous text or data sections, and AFAIK data always follows text.

a bin file doesn't have gaps and you need to specify where to put it in memory
an intelhex file can have gaps and specify the address for each line

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
« Last Edit: July 29, 2021, 09:18:25 pm by SiliconWizard »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #3 on: July 29, 2021, 09:27:38 pm »
I saw that post and thought he was trying to generate a an .exe (a DOS, or windoze command line executable). These have a complicated format, with a load-time fixed-up table of jumps etc.

I am also very familiar with Intel hex. But I don't want to use that, due to the bulk. For a 1MB CPU you would need ~2MB intel hex.

So I guess the objective here is just a string of bytes representing the CPU FLASH, 0x08000000 onwards, as long as there is text+data. This must incidentally be what gets fed to the debugger too.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: ELF to binary for boot loader
« Reply #4 on: July 29, 2021, 09:31:41 pm »
I saw that post and thought he was trying to generate a an .exe (a DOS, or windoze command line executable). These have a complicated format, with a load-time fixed-up table of jumps etc.

I am also very familiar with Intel hex. But I don't want to use that, due to the bulk. For a 1MB CPU you would need ~2MB intel hex.

So I guess the objective here is just a string of bytes representing the CPU FLASH, 0x08000000 onwards, as long as there is text+data. This must incidentally be what gets fed to the debugger too.

hex is a very simple format to deal with, much simpler than elf. So point is, you can use objcopy to generate a hex file from a elf file, and then do whatever you find convenient from the hex file. If a hex file is too big, you could either compress it, or convert it to some custom format.
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: ELF to binary for boot loader
« Reply #5 on: July 29, 2021, 09:33:57 pm »
I think you are confused. IDE issues very simple programming commands for the flash controller to the debugger/programmer, it does not  store anything.

Normally ELF files will be contiguous, unless you go out of your way to make them otherwise. And if you do, you need to come up with your own binary format that includes separate sections.
Alex
 

Offline ajb

  • Super Contributor
  • ***
  • Posts: 2604
  • Country: us
Re: ELF to binary for boot loader
« Reply #6 on: July 29, 2021, 09:53:32 pm »
You can certainly use objcopy to create a binary of the application and hand that off to a bootloader for installation.  You'll want to provide at least a length for it and maybe some other metadata so it's probably a good idea to define some sort of header the bootloader can inspect, and a checksum or cryptographic signature would be a really good idea.  Easy enough to create a script to build all of that into an update package.  There are even tools to script the parsing of source files, so you can do things like extract version information or other metadata from the application files and put that into the update package for easy automated deployment.  Of course if you don't have enough memory available to hold an entire binary image+header in addition to the installed software--whether in some sort of on-chip/soldered-on storage or a removable memory card or USB drive--then you'll need to drip-feed the binary to the bootloader from some other device.  What that other device is would be up to you/your application.
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: ELF to binary for boot loader
« Reply #7 on: July 30, 2021, 01:40:49 am »
Quote
You can certainly use objcopy to create a binary of the application and hand that off to a bootloader for installation.
And you probably should, because the elf format itself is relatively complex and probably subject to changing.

There is also readelf (a program) and libelf (a library for parsing elf files.)
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: ELF to binary for boot loader
« Reply #8 on: July 30, 2021, 01:56:34 am »
Parsing ELF just to extract the programming data is trivial. I actually prefer doing that over parsing HEX files, since there is no need to deal with a ton of formatting options and parsing text files.

But for the bootloaders, it makes no sense to directly accept either of them, you really want some dedicated format, possibly with check sums and stuff like that.
Alex
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: ELF to binary for boot loader
« Reply #9 on: July 30, 2021, 03:32:16 am »
I am also very familiar with Intel hex. But I don't want to use that, due to the bulk. For a 1MB CPU you would need ~2MB intel hex.

What is the problem with that?
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: ELF to binary for boot loader
« Reply #10 on: July 30, 2021, 03:37:12 am »
Parsing ELF just to extract the programming data is trivial. I actually prefer doing that over parsing HEX files, since there is no need to deal with a ton of formatting options and parsing text files.

Parsing Intel hex is easy. I do it in 50 lines for a simple RV32I emulator here:

https://github.com/brucehoult/trv/blob/main/trv.c

I'm sure I couldn't load an elf file with as little code -- certainly using my self-imposed no headers, no libraries constraint.

I'd be happy to be proven wrong.
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: ELF to binary for boot loader
« Reply #11 on: July 30, 2021, 05:06:30 am »
Here is 50 lines for parsing ELF files. And most of this is just structure definitions. The actual parsing is 20 lines.
Code: [Select]
typedef struct
{
  uint8_t      e_ident[16];
  uint8_t      e_type;
  uint16_t     e_machine;
  uint16_t     e_version;
  uint32_t     e_entry;
  uint32_t     e_phoff;
  uint32_t     e_shoff;
  uint32_t     e_flags;
  uint16_t     e_ehsize;
  uint16_t     e_phentsize;
  uint16_t     e_phnum;
  uint16_t     e_shentsize;
  uint16_t     e_shnum;
  uint16_t     e_shstrndx;
} elf_ehdr_t;

typedef struct
{
  uint32_t     sh_name;
  uint32_t     sh_type;
  uint32_t     sh_flags;
  uint32_t     sh_addr;
  uint32_t     sh_offset;
  uint32_t     sh_size;
  uint32_t     sh_link;
  uint32_t     sh_info;
  uint32_t     sh_addralign;
  uint32_t     sh_entsize;
} elf_shdr_t;

bool read_elf(uint8_t *data)
{
  elf_ehdr_t *hdr = (elf_ehdr_t *)data;
  elf_shdr_t *shdr = (elf_shdr_t *)&data[hdr->e_shoff];

  if (hdr->e_type != 2/*ET_EXEC*/ || hdr->e_version != 1/*EV_CURRENT*/ || hdr->e_machine != 40/*EM_ARM*/)
    return false;

  for (int i = 0; i < hdr->e_shnum; i++)
  {
    if (shdr[i].sh_type == 1/*SHT_PROGBITS*/ && (shdr[i].sh_flags & 2/*SHF_ALLOC*/))
    {
      printf("program %d bytes from file offset %d into memory address %d (0x%08x)\n",
          shdr[i].sh_size, shdr[i].sh_offset, shdr[i].sh_addr, shdr[i].sh_addr);
    }
  }

  return true;
}

Obviously replace printf() with the actual data extraction and programming functions.

This does not verify input data and valid ELF file is assumed.

And of course replace 40 (EM_ARM) in e_machine check with an actual target machine type. Or just remove the check and assume that ELF file is valid for the used processor.
« Last Edit: July 30, 2021, 05:09:54 am by ataradov »
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #12 on: July 30, 2021, 06:53:25 am »
You can certainly use objcopy to create a binary of the application and hand that off to a bootloader for installation.  You'll want to provide at least a length for it and maybe some other metadata so it's probably a good idea to define some sort of header the bootloader can inspect, and a checksum or cryptographic signature would be a really good idea.  Easy enough to create a script to build all of that into an update package.  There are even tools to script the parsing of source files, so you can do things like extract version information or other metadata from the application files and put that into the update package for easy automated deployment.  Of course if you don't have enough memory available to hold an entire binary image+header in addition to the installed software--whether in some sort of on-chip/soldered-on storage or a removable memory card or USB drive--then you'll need to drip-feed the binary to the bootloader from some other device.  What that other device is would be up to you/your application.

That's exactly what I need to do - many thanks. Just a straight binary image of the CPU FLASH which can be programmed into it directly.

Sounds like objcopy -O binary.dat will do this.

Then I will need to get a little executable written which packages that binary block with a length value, a CRC and maybe some crypto hash. Has anyone done this within Cube? AIUI it needs to be written in Java, unless you are happy for a Windows-only setup.
« Last Edit: July 30, 2021, 07:02:14 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #13 on: July 30, 2021, 07:22:20 am »
But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening.

Exactly this is happening. My understanding is that objcopy copies all loadable sections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied.

Quote from: man page
       objcopy can be used to generate a raw binary file by using an output
       target of binary (e.g., use -O binary).  When objcopy generates a raw
       binary file, it will essentially produce a memory dump of the contents
       of the input object file. All symbols and relocation information will
       be discarded. The memory dump will start at the load address of the
       lowest section copied into the output file.
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: ELF to binary for boot loader
« Reply #14 on: July 30, 2021, 07:51:49 am »
Here is 50 lines for parsing ELF files. And most of this is just structure definitions. The actual parsing is 20 lines.

Thanks, I'll give that a try.

I assume there's supposed to be padding after e_type and e_version? Otherwise all the 16 bit and 32 bit fields are unaligned. It woul dbe better if padding had explicit fields.

This can be handy for elf-in-ROM on small machines, or memory-mapped on big machines. Seems like potential for trouble if it wants to unpack over the top of the place you read it into...
 
The following users thanked this post: SiliconWizard

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #15 on: July 30, 2021, 07:56:15 am »
"Exactly this is happening. My understanding is that objcopy copies all loadable sections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied."

Fortunately this isn't a problem in my project.

I do have a bit of code which is linked to execute in RAM, halfway up ie 0x20010000, but it should not be loading that at that address.

It was discussed here, and is working fine
https://www.eevblog.com/forum/microcontrollers/how-to-create-elf-file-which-contains-the-normal-prog-plus-a-relocatable-block/new/#new

"IDE issues very simple programming commands for the flash controller to the debugger/programmer, it does not  store anything."

OK; that makes sense. That's why the more expensive debuggers claim to run faster. The debugger is probably programming every byte as a separate transaction. Although one should still be limited by the FLASH programming speed. STLINK V3 does appear to program a bit faster than V2. I think the only real advantage of a fast debugger is with stuff like breakpoints with an ignore count; these trigger a breakpoint on every hit and a fast debugger will be able to process this faster so slowing down the target less.

Is there any way to incorporate objcopy -O binary.dat into the Cube script, so that along the .elf (which is used by the debugger) it pops out the binary?
« Last Edit: July 30, 2021, 08:01:41 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: ELF to binary for boot loader
« Reply #16 on: July 30, 2021, 08:19:44 am »
I assume there's supposed to be padding after e_type and e_version? Otherwise all the 16 bit and 32 bit fields are unaligned.
Newer versions of the standard define e_type as uint16_t and e_version as uint32_t. Makes sense, I guess. This code is pretty old, I don't even remember where I've got this layout from.

This can be handy for elf-in-ROM on small machines, or memory-mapped on big machines.
Even stripped ELF files contain a lot more stuff, so if you are optimizing for size, it may not be the best idea to use ELF for anything.

Another issue is that for RAM code you don't really get a good info. ELF is not designed to be converted into binary, it was designed to be loaded as is. So even with this code you get output like this if you have RAM functions:
Quote
program 6884 bytes from file offset 65536 into memory address 4194304 (0x00400000)
program 4 bytes from file offset 72420 into memory address 4201188 (0x00401ae4)
program 4 bytes from file offset 72424 into memory address 4201192 (0x00401ae8)
program 112 bytes from file offset 131072 into memory address 541065216 (0x20400000)
0x00400000 in this case is the flash offset and 0x20400000 is SRAM. As you can see, the last allocated section is in the RAM, but the actual location of the data is in the flash.

The way objcopy handles this is it takes all the parts within the section (as defined by SECTIONS part of the linker script) and concatenates them together without filling the gaps. So in this case to simulate objcopy you would ignore the address and copy the data sequentially.

The only time objcopy fills the gaps is when there are multiple sections that contain loadable data. Those are the sections defined by the MEMORY part of the linker script. To extract that info add this:

Code: [Select]
typedef struct
{
  uint32_t     p_type;
  uint32_t     p_offset;
  uint32_t     p_vaddr;
  uint32_t     p_paddr;
  uint32_t     p_filesz;
  uint32_t     p_memsz;
  uint32_t     p_flags;
  uint32_t     p_align;
} elf_phdr_t;
..........

  elf_phdr_t *phdr = (elf_phdr_t *)&data[hdr->e_phoff];

  for (int i = 0; i < hdr->e_phnum; i++)
  {
    printf("--- %d\n", i);
    printf("p_type       : 0x%08x\n", phdr[i].p_type);
    printf("p_offset     : 0x%08x\n", phdr[i].p_offset);
    printf("p_vaddr      : 0x%08x\n", phdr[i].p_vaddr);
    printf("p_paddr      : 0x%08x\n", phdr[i].p_paddr);
    printf("p_filesz     : 0x%08x\n", phdr[i].p_filesz);
    printf("p_memsz      : 0x%08x\n", phdr[i].p_memsz);
    printf("p_flags      : 0x%08x\n", phdr[i].p_flags);
    printf("p_align      : 0x%08x\n", phdr[i].p_align);
  }

Although it looks like it may be relying on the physical address from the program headers as well to convert things:

Quote
Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x010000 0x00400000 0x00400000 0x01aec 0x01aec R E 0x10000
  LOAD           0x020000 0x20400000 0x00401aec 0x00070 0x081ac RWE 0x10000

 Section to Segment mapping:
  Segment Sections...
   00     .text .init .fini
   01     .data .bss

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 1] .text             PROGBITS        00400000 010000 001ae4 00  AX  0   0 64
  [ 2] .init             PROGBITS        00401ae4 011ae4 000004 00  AX  0   0  4
  [ 3] .fini             PROGBITS        00401ae8 011ae8 000004 00  AX  0   0  4
  [ 4] .data             PROGBITS        20400000 020000 000070 00 WAX  0   0  4

Here 0x00401aec is the address of the RAM functions (part of the .data segment) in the flash. If you just go by "Section Headers", then you will have a huge gap. But "Program Headers" alone do not have enough information.
« Last Edit: July 30, 2021, 08:34:45 am by ataradov »
Alex
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26906
  • Country: nl
    • NCT Developments
Re: ELF to binary for boot loader
« Reply #17 on: July 30, 2021, 08:21:51 am »
But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening.

Exactly this is happening. My understanding is that objcopy copies all loadable sections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied.
AFAIK you need to tell objcopy to fill the gaps. There is an option for that as well which also specifies the byte value to use.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: ELF to binary for boot loader
« Reply #18 on: July 30, 2021, 08:23:15 am »
That's why the more expensive debuggers claim to run faster.
They claim it to justify the price. Past certain point (at least using USB HS for transfers) there is no real difference. Most of the time is spent waiting for the flash to program anyway.

The debugger is probably programming every byte as a separate transaction.
No, because there are limitations on how you can program the flash. A debugger does the same exact things as you would from the code. So you can read the device datasheet and know exactly what debugger is doing. And it is not really a debugger doing those things, but the IDE. A debugger is just a pipe.

Some debuggers have more complicated logic they can run on the device itself. This is helpful for devices with poorly designed flash controllers (like ST) where fast programming is a time-sensitive operation and if the data is not delivered in time, the whole operation fails.

Is there any way to incorporate objcopy -O binary.dat into the Cube script, so that along the .elf (which is used by the debugger) it pops out the binary?
No idea. Usually it is done through post-build events.
« Last Edit: July 30, 2021, 08:38:09 am by ataradov »
Alex
 

Offline ajb

  • Super Contributor
  • ***
  • Posts: 2604
  • Country: us
Re: ELF to binary for boot loader
« Reply #19 on: July 30, 2021, 01:38:48 pm »
Is there any way to incorporate objcopy -O binary.dat into the Cube script, so that along the .elf (which is used by the debugger) it pops out the binary?


Cube IDE is based on eclipse, so unless they've really munged it up there should be a 'post build' section in the build configuration where you can tell it to invoke whatever external process you want.  Off the top of my head I don't remember where exactly. You can throw the objcopy command in there, and even invoke your packager from there was well. Post build steps are per build configuration, so you can have this all set up on a 'release' build config and leave your debug config as is since you probably don't need it there.

I'm not a huge fan of python in general but it's well suited to these sorts of things, being cross platform and having a ton of libraries. You could probably do a basic packager in <20 lines including hashing.
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #20 on: July 31, 2021, 07:51:23 am »
But there must be a way to specify gaps otherwise if you generated some text for bottom 100k and then top 40k, you would end up with a 1MB binary block, mostly empty. I don't think this is happening.

Exactly this is happening. My understanding is that objcopy copies all loadable sections in the .elf file to the .bin file. Gaps betwen sections are filled. Non-loadable sections (like debug, etc.) are not copied.

AFAIK you need to tell objcopy to fill the gaps. There is an option for that as well which also specifies the byte value to use.

$ objcopy -O binary a.out a.bin

$ ls -l a.bin
-rwxr-xr-x 1 fuer users 8288 Jul 31 09:35 a.bin

$ objdump -h a.out

a.out:     Dateiformat elf64-x86-64

Sektionen:
Idx Name          Size      VMA               LMA               File off  Algn
  0 text1         00000007  0000000000003000  0000000000003000  00003000  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 text2         00000007  0000000000005000  0000000000005000  00005000  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .eh_frame     00000058  0000000000005008  0000000000005008  00005008  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .comment      00000029  0000000000000000  0000000000000000  00005060  2**0
                  CONTENTS, READONLY


The size of a.bin is 8288 = LMA(.eh_frame) + SIZEOF(.eh_frame) - LMA(text1).
Obviously it does keep and fill the gap between text1 and text2, and between text2 and .eh_frame, preserving the offsets beween the copied sctions' load addresses also in the .bin file.
« Last Edit: July 31, 2021, 07:55:08 am by gf »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #21 on: July 31, 2021, 08:23:40 am »
Thank you everyone. This is great. Yes the post build steps can be found under Project / Properties



I am going to put a job up on freelancer.com to get a win32 command line utility done which appends four crc32 bytes, little-endian, to a file supplied on a command line :) I see some other IDE (IAR, I think) offers this as a built-in function. Or maybe it's been done before... well it must have been.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: ELF to binary for boot loader
« Reply #22 on: July 31, 2021, 09:22:06 am »
I am going to put a job up on freelancer.com to get a win32 command line utility done which appends four crc32 bytes, little-endian, to a file supplied on a command line :) I see some other IDE (IAR, I think) offers this as a built-in function. Or maybe it's been done before... well it must have been.

On Mac or Linux the following ...

Code: [Select]
perl -e 'print pack("V",int($ARGV[0])),"\n"' $(cksum foo) >>foo

... will literally append crc32 to foo, little endian. As in modify the given file. If you want big endian change "V" (Vax) to "N" (Network).

On Windows you can (and should) install Linux in WSL.
 

Offline abyrvalg

  • Frequent Contributor
  • **
  • Posts: 824
  • Country: es
Re: ELF to binary for boot loader
« Reply #23 on: July 31, 2021, 09:25:48 am »
A Python tool appending CRC32 matching STM32 hw CRC:
Code: [Select]
#!/usr/bin/python3
'''CRC32 matching STM32 hardware algo'''

_table = {}
_poly = 0x04C11DB7
for i in range(256):
    c = i << 24

    for j in range(8):
        c = (c << 1) ^ _poly if (c & 0x80000000) else c << 1

    _table[i] = c & 0xffffffff


def crc32stm(buf, prev=0xFFFFFFFF):
    crc = prev
    for i in range(0, len(buf), 4):
        crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+3]]
        crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+2]]
        crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i+1]]
        crc = ((crc << 8) & 0xffffffff) ^ _table[(crc >> 24) ^ buf[i]]
    return crc


from sys import argv, exit
from struct import pack


if len(argv)!=2:
    exit("Usage: "+argv[0]+" <file>")

with open(argv[1], 'rb+') as fp:
    crc = crc32stm(fp.read())
    fp.write(pack('<I', crc))

To use in Cube:
- install Python 3 (enable "install launcher" and "add to PATH")
- add post-build command: python crc32stm.py yourbinary.bin   
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4037
  • Country: nz
Re: ELF to binary for boot loader
« Reply #24 on: July 31, 2021, 09:51:37 am »
Oh crud .. cksum for some reason appends the length of the file to the data before calculating the CRC. That sucks.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #25 on: August 01, 2021, 06:41:52 am »
Here is the source for a win32 .exe which appends the CRC32 (the inverted JAMCRC) to a file:

Code: [Select]
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>


/*
 *
 * CRC-32/JAMCRC. This is a "rolling" algorithm. Invert the *final* result for ISO-HDLC.
 * Returns 0x098494f3 from "123456789" (9 bytes). Inverting this (at the end) gives 0xcbf43926.
 * Polynomial is 0x04c11db7 and it holds it backwards (0xEDB88320).
 * This version accepts one byte at a time, and maintains the CRC in crcvalue. This makes it suitable
 * for calculating a CRC across a number of data blocks.
 * Speed is approx 400kbytes/sec.
 * crcvalue must be initialised to 0xffffffff by caller, and holds the accumulated CRC as you go along
 *
 */


void crc32(uint8_t input_byte, uint32_t *crcvalue)
{
for (uint32_t j = 0; j < 8; j++)
{
uint32_t mask = (input_byte^*crcvalue) & 1;
*crcvalue >>= 1;
if (mask)
*crcvalue = *crcvalue ^ 0xEDB88320;
input_byte >>= 1;
}
}

int main(int argc, char* argv[])
{
uint32_t crcvalue = 0xffffffff;
FILE* file = NULL;
if (argc > 1)
{
if (argc > 2)
{
int c = 0;
if (!_strnicmp(argv[2], "0x", 2))
c = 2;
crcvalue = strtol(&argv[2][c], NULL, 16);
}
file = fopen(argv[1], "rb");
if (file)
{
while (!feof(file))
{
uint8_t input_byte;
if (fread(&input_byte, sizeof(input_byte), 1, file))
{
crc32(input_byte, &crcvalue);
}
//else
//{
// printf("Before last zero byte = %08X\r\n", crcvalue);
// crc32(0, &crcvalue);
// printf("After last zero byte = %08X\r\n", crcvalue);
//}
}
fclose(file);

crcvalue = ~crcvalue;
file = fopen(argv[1], "ab");
fwrite(&crcvalue, sizeof(uint32_t), 1, file);
fclose(file);
}
else
{
printf("Cannot open file\r\n");
}

file = NULL;
}
else
{
printf("Missing file name\r\n");
}

return 0;
}


An optional 2nd parameter is a different CRC initialisation value (default is 0xffffffff).

Just curious: someone said that anything running out of Cube IDE should be written in Java.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #26 on: August 08, 2021, 12:46:27 pm »
I have finished the boot loader and it is programming the flash fine. I will post my code as an update to the other thread... took a few days to get it right.

However I have problems producing the .bin file (to which I am appending the CRC). So currently I am flashing various jpeg files, which obviously don't run too well ;)

The batch file I am using, in the post build steps configuration, and which runs with no errors, is:

Code: [Select]
cd c:\kde420\project1\debug
arm-none-eabi-objdump -h kde2020.elf > objdump.txt
arm-none-eabi-objcopy --strip-debug -O binary kde2020.elf c:\kde420\project1\kde420.bin
cd c:\kde420\project1\
c:\kde420\crcgen\addcrc kde420.bin 0xffffffff


I noticed Cube is already generating a .bin file but that one is also 131MB. Hence I am generating my different kde420.bin file, but that is no smaller.

Clearly objcopy is including stuff which is not in the final program (which is about 240k). The objdump.txt I am creating contains this

Code: [Select]
kde2020.elf:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 loader_bss    00000000  2001064c  2001064c  00090000  2**0
                  CONTENTS
  1 KDE_loader    0000064c  20010000  08002c48  00030000  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .isr_vector   00000188  08000000  08000000  00010000  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .b_main.o     000011a4  08000188  08000188  00010188  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  4 .45dbxx.o     000009b0  0800132c  0800132c  0001132c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  5 .KDE_loader.o 00000000  08001cdc  08001cdc  00090000  2**0
                  CONTENTS
  6 .main_stub.o  00000008  08008000  08008000  00038000  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  7 .main.o       000002c0  08008008  08008008  00038008  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  8 .text         000382c0  080082d0  080082d0  000382d0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  9 .ARM.exidx    00000008  08040590  08040590  00070590  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .init_array   00000008  08040598  08040598  00070598  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 11 .fini_array   00000004  080405a0  080405a0  000705a0  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 12 .data         00000f6c  20000000  08001cdc  00020000  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 13 .KDE_loader   0000064c  08002c48  08002c48  00022c48  2**0
                  ALLOC
 14 .bss          0000ddac  20000f70  20000f70  00090f70  2**3
                  ALLOC
 15 .main_heap    00000004  2000ed1c  2000ed1c  00090f70  2**0
                  ALLOC
 16 .ccmram       00010000  10000000  10000000  00080000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 17 .ARM.attributes 00000030  00000000  00000000  00090000  2**0
                  CONTENTS, READONLY
 18 .debug_info   00079159  00000000  00000000  00090030  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 19 .debug_abbrev 00010d81  00000000  00000000  00109189  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 20 .debug_aranges 00003eb8  00000000  00000000  00119f10  2**3
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 21 .debug_ranges 00003948  00000000  00000000  0011ddc8  2**3
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 22 .debug_macro  000417ae  00000000  00000000  00121710  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 23 .debug_line   00062b3d  00000000  00000000  00162ebe  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 24 .debug_str    001128f3  00000000  00000000  001c59fb  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 25 .comment      00000053  00000000  00000000  002d82ee  2**0
                  CONTENTS, READONLY
 26 .debug_frame  0000d4c4  00000000  00000000  002d8344  2**2
                  CONTENTS, READONLY, DEBUGGING, OCTETS

It looks like I should have only this (I do want initialised DATA included, obviously)



Any ideas on how to achieve this?

I've been looking at various references e.g. https://sourceware.org/binutils/docs/binutils/objcopy.html but none of the --strip... options I have tried make any difference.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #27 on: August 08, 2021, 01:19:52 pm »
My understanding is that all sections tagged with "LOAD" are included in the bin file, and all gaps between the load addresses of the included sections are included, too, and filled.
The section .ccmram is tagged with "LOAD" too, and there seems to be a larger gap between .data and .ccram.

Edit: So if I did not overlook anything, I'd expect the .bin file to contain the load address range 0x08000000 - 0x1000ffff (which implies a file size of 134283264 bytes)
« Last Edit: August 08, 2021, 01:47:17 pm by gf »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #28 on: August 08, 2021, 01:53:47 pm »
Yeah... after much hacking I found this gives me a 258k file

arm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary kde2020.elf output.bin

which seems to contain the right stuff - at least in the beginning :)

FWIW I am running the Cube post build stuff out of a batch file. I didn't find any way to add multiple command lines. Some people online said they did but Cube didn't preserve their order, but putting them in a .bat sidestepped that.

AND IT IS RUNNING THE CODE! :) Took a few days' coding.
« Last Edit: August 08, 2021, 02:05:24 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #29 on: August 08, 2021, 02:00:47 pm »
Sure, with -R you can exclude sections. But the actual question is, why did .ccram become a loadable section in the first place, if it is not suposed to become part of the flash? I guess this is a bug in the linker script?
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #30 on: August 08, 2021, 02:10:05 pm »
Yes course there is a bug in my linkfile :) I am right on the edge with those.

I now need a reboot function, for the boot loader, which doesn't need calling any of the ST stuff. I condensed this out of one of the HAL... things but it just hangs. I will work on it but maybe somebody knows something right off

Code: [Select]
static void reboot(void)
{
  __ASM volatile ("dsb 0xF":::"memory"); // Ensure all outstanding memory accesses included
                                                // buffered write are completed before reset
  SCB->AIRCR  = (uint32_t)((0x5FAUL << 16) | (SCB->AIRCR & 0x700) | 0 ); // Keep priority group unchanged
  __ASM volatile ("dsb 0xF":::"memory"); // Ensure completion of memory access

  for(;;) // wait until reset
  {
  __ASM volatile ("nop");
  }
}
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline cv007

  • Frequent Contributor
  • **
  • Posts: 826
Re: ELF to binary for boot loader
« Reply #31 on: August 08, 2021, 03:07:18 pm »
bit 3- SYSRESETREQ - needs to be set.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #32 on: August 08, 2021, 04:00:36 pm »
Nothing in the RM, strangely enough. This is in core_cm4.h and did eventually work. I originally tried to simplify it a bit - not a good idea :)

Code: [Select]
__NO_RETURN __STATIC_INLINE void __NVIC_SystemReset(void)
{
  __DSB();                                                          /* Ensure all outstanding memory accesses included
                                                                       buffered write are completed before reset */
  SCB->AIRCR  = (uint32_t)((0x5FAUL << SCB_AIRCR_VECTKEY_Pos)    |
                           (SCB->AIRCR & SCB_AIRCR_PRIGROUP_Msk) |
                            SCB_AIRCR_SYSRESETREQ_Msk    );         /* Keep priority group unchanged */
  __DSB();                                                          /* Ensure completion of memory access */

  for(;;)                                                           /* wait until reset */
  {
    __NOP();
  }
}


In case anyone is interested, this is my linkfile

Code: [Select]
/* Entry Point */
ENTRY(Reset_Handler)

/* Reference boot block stuff to ensure that it gets linked-in - not sure if these do anything */
/* EXTERN(b_boot) */
/* EXTERN(KDE_loader) */


/* Highest address of the main stack. This stack is only for startup code, boot loader, and ISRs. The RTOS has its own. */
 /*  _estack = 0x20020000;  */    /* stack in 128K RAM */
 _estack = 0x10010000;    /* stack in  64k CCM - note: configTOTAL_HEAP_SIZE + min_stack_size must not exceed 64k) */
 
/* top of RAM for _sbrk - top of heap check */
 _top = 0x20020000;

/* Heap and stack sizes */
 
_Min_Heap_Size  = 0x0000;     /* 21/7/21 set to zero to prevent heap/stack conflict check from blowing up due to boot loader area  */
_Min_Stack_Size = 0x4000;     /* 16k stack - in CCM */

/* Specify the memory areas */
/* CCMRAM added PH 12/5/2021 - cannot use with DMA */
MEMORY
{
FLASH_BOOT (rx)     : ORIGIN = 0x08000000, LENGTH = 32K
FLASH_APP (rx)      : ORIGIN = 0x08008000, LENGTH = 1024K-32K
RAM (xrw)           : ORIGIN = 0x20000000, LENGTH = 128K
MEMORY_B1 (rx)      : ORIGIN = 0x60000000, LENGTH = 0K
CCMRAM (rw)         : ORIGIN = 0x10000000, LENGTH = 64K
}

/* Define output sections */
SECTIONS
{
 
/* loader_bss and loader sections must come first, in order to override the
     wildcards in subsequent sections */
  loader_bss _loader_end :
  {
  _loader_bss_start = .;
  *KDE_loader.o(.bss*)
  . = ALIGN(4);
  _loader_bss_end = .;
}

/* General purpose buffer for loader usage, b_main.c and KDE_loader.c.
   Done this way to avoid spurious section overlap errors; the loader reboots when done
   so it can use any RAM it wants */
  buffer1=0x10000000; /* base of CCM */
 
  /* RAM based loader is linked to execute here */
  KDE_loader 0x20010000 : AT(_loader_loadaddr)
  {
  _loader_start = .;
  *KDE_loader.o(.text*)
  *KDE_loader.o(.rodata*)
  *KDE_loader.o(.data*)
  *KDE_loader.o(.common*)
  *KDE_loader.o(.ARM.attributes)
  . = ALIGN(4);
  _loader_end = .;
  }
 
  _loader_size = SIZEOF(KDE_loader);
 


  /* Place modules into FLASH, starting at the bottom (0x08000000)
     The line "*b_main.o (.text .text* .rodata .rodata*)" is important, but for some reason
     (probably the reset_handler reference) is not needed for isr_vector */
 
   
    /* Startup code */
  .isr_vector :
  {
    . = ALIGN(4);
    KEEP(*(.isr_vector))
    . = ALIGN(4);
  } >FLASH_BOOT
 
  .b_main.o :
  {
    . = ALIGN(4);
    KEEP(*(.b_main.o))
    *b_main.o (.text .text* .rodata .rodata*)
    . = ALIGN(4);
  } >FLASH_BOOT
 
  .45dbxx.o :
  {
    . = ALIGN(4);
    KEEP(*(.45dbxx.o))
    *45dbxx.o (.text .text* .rodata .rodata*)
    . = ALIGN(4);
  } >FLASH_BOOT
 
  /* === other boot block stuff will go here === */
 
 
  /* KDE_loader.o - goes last in the bottom block which makes it easier to check the block usage
     by checking _loader_end_in_flash in the .map file */
 
  .KDE_loader.o :
  {
    . = ALIGN(4);
    KEEP(*(.KDE_loader.o))
    *KDE_loader.o (.text .text* .rodata .rodata*)
    . = ALIGN(4);
      _loader_end_in_flash = .;
  } >FLASH_BOOT
 
  /* The rest of the KDE code goes here, loaded at base+32k, starting with a stub and then the real main() */

  .main_stub.o :
  {
    . = ALIGN(4);
    KEEP(*(.main_stub.o))
    *main_stub.o (.text .text* .rodata .rodata*)
    . = ALIGN(4);
  } >FLASH_APP
   
  .main.o :
  {
    . = ALIGN(4);
    KEEP(*(.main.o))
    *main.o (.text .text* .rodata .rodata*)
    . = ALIGN(4);
  } >FLASH_APP
 
/* This collects all other stuff, which gets loaded into FLASH after main.o above */
 
  .text :
  {
    . = ALIGN(4);
    *(.text)           /* .text sections (code) */
    *(.text*)          /* .text* sections (code) */
    *(.rodata)         /* .rodata sections (constants, strings, etc.) */
    *(.rodata*)        /* .rodata* sections (constants, strings, etc.) */
    *(.glue_7)         /* glue arm to thumb code */
    *(.glue_7t)        /* glue thumb to arm code */
*(.eh_frame)

    KEEP (*(.init))
    KEEP (*(.fini))

    . = ALIGN(4);
    _etext = .;        /* define a global symbol at end of code */
} >FLASH_APP

/* this is for __libc_init_array() */

.preinit_array     :
  {
    PROVIDE_HIDDEN (__preinit_array_start = .);
    KEEP (*(.preinit_array*))
    PROVIDE_HIDDEN (__preinit_array_end = .);
  } >FLASH_APP
  .init_array :
  {
    PROVIDE_HIDDEN (__init_array_start = .);
    KEEP (*(SORT(.init_array.*)))
    KEEP (*(.init_array*))
    PROVIDE_HIDDEN (__init_array_end = .);
  } >FLASH_APP
  .fini_array :
  {
    PROVIDE_HIDDEN (__fini_array_start = .);
    KEEP (*(.fini_array*))
    KEEP (*(SORT(.fini_array.*)))
    PROVIDE_HIDDEN (__fini_array_end = .);
  } >FLASH_APP


/* Initialized data sections goes into RAM, load LMA copy after code */
  .data :
  {
    . = ALIGN(4);
    _sdata = .;        /* create a global symbol at data start */
    *(.data)           /* .data sections */
    *(.data*)          /* .data* sections */
      . = ALIGN(4);
    _edata = .;        /* define a global symbol at data end */
  } >RAM  AT >FLASH_BOOT

  /* used by the startup to initialize data */
  _sidata = LOADADDR(.data);

/* dummy placeholder in flash for loader section, to count flash usage */
  .KDE_loader :
  {
  . = . + SIZEOF(KDE_loader);
  } AT >FLASH_BOOT
 
  _loader_loadaddr = LOADADDR(.KDE_loader);
 
  /* Uninitialized data section */
  . = ALIGN(4);
  .bss :
  {
    /* This is used by the startup in order to initialize the .bss section */
    _sbss = .;         /* define a global symbol at bss start */
    __bss_start__ = _sbss;
    *(.bss)
    *(.bss*)
    *(COMMON)

    . = ALIGN(4);
    _ebss = .;         /* define a global symbol at bss end */
    __bss_end__ = _ebss;
  } >RAM

  /* The heap ends up after BSS in main RAM */
  /* This also checks that the top of the heap doesn't hit the bottom of the stack i.e. how much RAM left */
  /* User_heap_stack section, used to check that there is enough RAM left */
 
  /* ._user_heap_stack : */
  /* This check is basically disabled because _Min_Heap_Size=0 above */
  .main_heap :
  {
    . = ALIGN(8);
    PROVIDE ( end = . );
    PROVIDE ( _end = . );
    . = . + _Min_Heap_Size;
  /*   . = . + _Min_Stack_Size; */  /* PH 14/5/2021 stack is in CCM, not here */
    . = ALIGN(8);
} >RAM
  /*   } >CCMRAM */


/* MEMORY_bank1 section, code must be located here explicitly            */
/* Example: extern int foo(void) __attribute__ ((section (".mb1text"))); */
/* Not used 14/5/2021 - was apparently used for LCD display on ST dev kit */
  .memory_b1_text :
  {
    *(.mb1text)        /* .mb1text sections (code) */
    *(.mb1text*)       /* .mb1text* sections (code)  */
    *(.mb1rodata)      /* read-only data (constants) */
    *(.mb1rodata*)
} >MEMORY_B1
 
/* CCM-RAM section
  *
  * IMPORTANT NOTE!
  * If variables placed in this section must be zero initialized,
  * the startup code needs to be modified to initialize this section.
  * Done PH 12/5/2021
  */
  .ccmram :
  {
    . = ALIGN(4);
    _sccmram = .;       /* create a global symbol at ccmram start */
    *(.ccmram)
    *(.ccmram*)
   
    . = ALIGN(4);
    _eccmram = .;       /* create a global symbol at ccmram end */
  } >CCMRAM

  /* Remove information from the standard libraries */
  /DISCARD/ :
  {
    libc.a ( * )
    libm.a ( * )
    libgcc.a ( * )
  }

  .ARM.attributes 0 : { *(.ARM.attributes) }
}


When I get around to it I will post my flashing code here
https://www.eevblog.com/forum/microcontrollers/32f417-best-way-to-program-the-flash-from-ram-based-code/new/#new

A lot of work went into it.
« Last Edit: August 08, 2021, 04:22:50 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ajb

  • Super Contributor
  • ***
  • Posts: 2604
  • Country: us
Re: ELF to binary for boot loader
« Reply #33 on: August 09, 2021, 06:26:56 am »
The reference manual for ARM MCUs only covers the information specific to that part: bus architecture, memory layout, peripherals, that sort of thing.  The operation of the core, of which the NVIC is part, is going to be the same as any other part with the same core (notwithstanding options like caches, FPU, etc).  So there's no point in including those details because they're already adequate covered by the ARM architecture reference material--and being the ones who designed the core, ARM are in a better position to document that stuff anyway. 

So for a comprehensive understanding of an ARM part you really the datasheet for electrical/performance, the reference manual for operational and peripheral information, and the relevant ARM documentation for how the core works
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #34 on: August 09, 2021, 11:15:03 am »
Quote
Sure, with -R you can exclude sections. But the actual question is, why did .ccram become a loadable section in the first place, if it is not suposed to become part of the flash? I guess this is a bug in the linker script?

Did you possibly place any initialized data in your .ccram section?
Seems that the section does have some contents:

Code: [Select]
16 .ccmram       00010000  10000000  10000000  00080000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #35 on: August 09, 2021, 12:08:54 pm »
There should be nothing initialised in the CCM.

The CCM is 64k, which is allocated as

48k for the FreeRTOS "buffer" (it runs all the processes, including their stacks, in there) and 16k general stack (used by main() before the RTOS starts, and by ISRs, and of course by the boot loader which basically gets control instead of (most of) main().

I have no variables like
uint64_t fred=55;
in the CCM.

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #36 on: August 09, 2021, 01:17:30 pm »
Hmm. Tried a small test pogram:

Code: [Select]
char buffer[1024] __attribute__((section(".ccram")));


Unllike regular uninitialized global variables (which either become common, or go into bss w/o "CONTENTS" or "LOAD" tag), gcc obviously does initialize buffer[] with zeros.

Code: [Select]
$ objdump -h -s x.o

Sektionen:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         00000000  0000000000000000  0000000000000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .data         00000000  0000000000000000  0000000000000000  00000040  2**0
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00000000  0000000000000000  0000000000000000  00000040  2**0
                  ALLOC
  3 .ccram        00000400  0000000000000000  0000000000000000  00000040  2**5
                  CONTENTS, ALLOC, LOAD, DATA
  4 .comment      0000002a  0000000000000000  0000000000000000  00000440  2**0
                  CONTENTS, READONLY
  5 .note.GNU-stack 00000000  0000000000000000  0000000000000000  0000046a  2**0
                  CONTENTS, READONLY
Inhalt von Abschnitt .ccram:
 0000 00000000 00000000 00000000 00000000  ................
 0010 00000000 00000000 00000000 00000000  ................
 0020 00000000 00000000 00000000 00000000  ................
 0030 00000000 00000000 00000000 00000000  ................
 0040 00000000 00000000 00000000 00000000  ................
 0050 00000000 00000000 00000000 00000000  ................
 0060 00000000 00000000 00000000 00000000  ................
 0070 00000000 00000000 00000000 00000000  ................
 0080 00000000 00000000 00000000 00000000  ................
 0090 00000000 00000000 00000000 00000000  ................
 00a0 00000000 00000000 00000000 00000000  ................
 00b0 00000000 00000000 00000000 00000000  ................
 00c0 00000000 00000000 00000000 00000000  ................
 00d0 00000000 00000000 00000000 00000000  ................
 00e0 00000000 00000000 00000000 00000000  ................
 00f0 00000000 00000000 00000000 00000000  ................
 0100 00000000 00000000 00000000 00000000  ................
 0110 00000000 00000000 00000000 00000000  ................
 0120 00000000 00000000 00000000 00000000  ................
 0130 00000000 00000000 00000000 00000000  ................
 0140 00000000 00000000 00000000 00000000  ................
 0150 00000000 00000000 00000000 00000000  ................
 0160 00000000 00000000 00000000 00000000  ................
 0170 00000000 00000000 00000000 00000000  ................
 0180 00000000 00000000 00000000 00000000  ................
 0190 00000000 00000000 00000000 00000000  ................
 01a0 00000000 00000000 00000000 00000000  ................
 01b0 00000000 00000000 00000000 00000000  ................
 01c0 00000000 00000000 00000000 00000000  ................
 01d0 00000000 00000000 00000000 00000000  ................
 01e0 00000000 00000000 00000000 00000000  ................
 01f0 00000000 00000000 00000000 00000000  ................
 0200 00000000 00000000 00000000 00000000  ................
 0210 00000000 00000000 00000000 00000000  ................
 0220 00000000 00000000 00000000 00000000  ................
 0230 00000000 00000000 00000000 00000000  ................
 0240 00000000 00000000 00000000 00000000  ................
 0250 00000000 00000000 00000000 00000000  ................
 0260 00000000 00000000 00000000 00000000  ................
 0270 00000000 00000000 00000000 00000000  ................
 0280 00000000 00000000 00000000 00000000  ................
 0290 00000000 00000000 00000000 00000000  ................
 02a0 00000000 00000000 00000000 00000000  ................
 02b0 00000000 00000000 00000000 00000000  ................
 02c0 00000000 00000000 00000000 00000000  ................
 02d0 00000000 00000000 00000000 00000000  ................
 02e0 00000000 00000000 00000000 00000000  ................
 02f0 00000000 00000000 00000000 00000000  ................
 0300 00000000 00000000 00000000 00000000  ................
 0310 00000000 00000000 00000000 00000000  ................
 0320 00000000 00000000 00000000 00000000  ................
 0330 00000000 00000000 00000000 00000000  ................
 0340 00000000 00000000 00000000 00000000  ................
 0350 00000000 00000000 00000000 00000000  ................
 0360 00000000 00000000 00000000 00000000  ................
 0370 00000000 00000000 00000000 00000000  ................
 0380 00000000 00000000 00000000 00000000  ................
 0390 00000000 00000000 00000000 00000000  ................
 03a0 00000000 00000000 00000000 00000000  ................
 03b0 00000000 00000000 00000000 00000000  ................
 03c0 00000000 00000000 00000000 00000000  ................
 03d0 00000000 00000000 00000000 00000000  ................
 03e0 00000000 00000000 00000000 00000000  ................
 03f0 00000000 00000000 00000000 00000000  ................
Inhalt von Abschnitt .comment:
 0000 00474343 3a202855 62756e74 7520372e  .GCC: (Ubuntu 7.
 0010 352e302d 33756275 6e747531 7e31382e  5.0-3ubuntu1~18.
 0020 30342920 372e352e 3000               04) 7.5.0.


Edit: I wonder whether

    .ccmram (NOLOAD) :
    {
        ...
    }


in the linker script helps?
« Last Edit: August 09, 2021, 01:37:35 pm by gf »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #37 on: August 09, 2021, 07:46:08 pm »
Do you mean that if one allocates say 16k of CCM to a buffer, you end up with a 16k block of 0x00 in the initialised DATA?

It wouldn't actually get copied to CCM because my startup (the st... .s bit) doesn't have any code for that.

I excluded that section from the binary output anyway, with

arm-none-eabi-objcopy -S --strip-all -R .bss -R .main_heap -R .ccmram -O binary kde2020.elf c:\kde420\project1\factory0.dat

so I am ok with it... but this was probably responsible for the 131MB bin file :)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #38 on: August 09, 2021, 08:09:04 pm »
Quote
Do you mean that if one allocates say 16k of CCM to a buffer, you end up with a 16k block of 0x00 in the initialised DATA?

Obviously, but the 16k of zeros are stored as contents of the .ccram section, not in the .data section.
Just do an objdump -h -s on the .o file, in order to check.

Quote
but this was probably responsible for the 131MB bin file

In your objdump output, the section .ccram is tagged with LOAD, therefore it is included in the .bin file (unless you exclude it explicitly).
And if both, .data and .ccram are included, then the huge gap between .data and .ccram is included as well in the .bin file.
The .bin file represents a memory dump of contiguous load addresses, so any gaps between incuded sections have to be filled.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: ELF to binary for boot loader
« Reply #39 on: August 09, 2021, 09:38:49 pm »
"Just do an objdump -h -s on the .o file, in order to check."

This is the section list

Code: [Select]
project1.elf:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 loader_bss    00000000  20010860  20010860  00090000  2**0
                  CONTENTS
  1 KDE_loader    00000860  20010000  08002c98  00030000  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  2 .isr_vector   00000188  08000000  08000000  00010000  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .b_main.o     000011f4  08000188  08000188  00010188  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  4 .45dbxx.o     000009b0  0800137c  0800137c  0001137c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  5 .KDE_loader.o 00000000  08001d2c  08001d2c  00090000  2**0
                  CONTENTS
  6 .main_stub.o  00000008  08008000  08008000  00038000  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  7 .main.o       000002c0  08008008  08008008  00038008  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  8 .text         000383f0  080082d0  080082d0  000382d0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  9 .ARM.exidx    00000008  080406c0  080406c0  000706c0  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .init_array   00000008  080406c8  080406c8  000706c8  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 11 .fini_array   00000004  080406d0  080406d0  000706d0  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 12 .data         00000f6c  20000000  08001d2c  00020000  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 13 .KDE_loader   00000860  08002c98  08002c98  00022c98  2**0
                  ALLOC
 14 .bss          0000ddac  20000f70  20000f70  00090f70  2**3
                  ALLOC
 15 .main_heap    00000004  2000ed1c  2000ed1c  00090f70  2**0
                  ALLOC
 16 .ccmram       00010000  10000000  10000000  00080000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 17 .ARM.attributes 00000030  00000000  00000000  00090000  2**0
                  CONTENTS, READONLY
 18 .debug_info   0007935a  00000000  00000000  00090030  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 19 .debug_abbrev 00010da6  00000000  00000000  0010938a  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 20 .debug_aranges 00003ec0  00000000  00000000  0011a130  2**3
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 21 .debug_ranges 00003938  00000000  00000000  0011dff0  2**3
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 22 .debug_macro  000417ae  00000000  00000000  00121928  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 23 .debug_line   00062dcb  00000000  00000000  001630d6  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 24 .debug_str    001128fe  00000000  00000000  001c5ea1  2**0
                  CONTENTS, READONLY, DEBUGGING, OCTETS
 25 .comment      00000053  00000000  00000000  002d879f  2**0
                  CONTENTS, READONLY
 26 .debug_frame  0000d4d4  00000000  00000000  002d87f4  2**2
                  CONTENTS, READONLY, DEBUGGING, OCTETS

I see a 64k CCM block there, not 16k. Could be this bit in the linkfile

Code: [Select]
MEMORY
{
FLASH_BOOT (rx)     : ORIGIN = 0x08000000, LENGTH = 32K
FLASH_APP (rx)      : ORIGIN = 0x08008000, LENGTH = 1024K-32K
RAM (xrw)           : ORIGIN = 0x20000000, LENGTH = 128K
MEMORY_B1 (rx)      : ORIGIN = 0x60000000, LENGTH = 0K
CCMRAM (rw)         : ORIGIN = 0x10000000, LENGTH = 64K
}

"The .bin file represents a memory dump of contiguous load addresses, so any gaps between incuded sections have to be filled."

Sure; I see that. However this wheel must have been re-invented by so many already, because sure as hell Cube is not sending 131MB to the STLINK debugger :) They must be doing "something".

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online gf

  • Super Contributor
  • ***
  • Posts: 1177
  • Country: de
Re: ELF to binary for boot loader
« Reply #40 on: August 09, 2021, 09:53:58 pm »
"Just do an objdump -h -s on the .o file, in order to check."
This is the section list

I meant the .o file where the ccram buffer is defined, not the final .elf.
The .map file generated by the linker may interesting too, as it tells in detail the input sections and .o files which have been included into each output section, and at which address.
 

Offline abyrvalg

  • Frequent Contributor
  • **
  • Posts: 824
  • Country: es
Re: ELF to binary for boot loader
« Reply #41 on: August 09, 2021, 09:58:03 pm »
.bss* section names are recognized by as/ld and treated as NOLOAD automatically (see GNU as manual section 4.2). You either need to specify NOLOAD type (as gf already advised, I would prefer this way too, explicit is better than implicit) or name it somehow like .bss.ccmram.
Manuals are cool :-+
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf