EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: HwAoRrDk on August 12, 2023, 03:13:52 pm

Title: Include a binary blob file into firmware executable at link-time
Post by: HwAoRrDk on August 12, 2023, 03:13:52 pm
Can anyone give me any pointers as to how I can include a binary blob file into my firmware executable at link-time using GCC? So that its contents can be referenced in code as if it was an array of bytes, like any other global variable. I also want to include and reference a CRC32 of the data as well.

At the moment I'm doing it the dumb way by converting my blob to a C array of hex constants, and then including that alongside the rest of my code at compile time. But I would like to be able to just update the file and have it included when building, without the intermediary step of conversion to C code.

I gather one way to do it is to use ld (or objcopy) to turn your blob into an object file and then link that to the executable, like so:

Code: [Select]
ld -r -b binary -o blob.o blob.bin

But then you just get three symbols available for _binary_blob_bin_start, _binary_blob_bin_end, and _binary_blob_bin_size. I suppose I could just use the 'start' symbol as my array:

Code: [Select]
extern uint8_t _binary_blob_bin_start[];

But there are questions I have about this approach:

- What is the, uh, size of 'size'? uint32_t? Something else?
- I would prefer if I have some control over what the names of the symbols are. Possible?
- What section of the executable does it get put in? .text? Normally, with my current approach it goes in .rodata.

More research turns up that there is also a way of including a blob file using directives in the linker script, but I don't really understand how that works, mostly because linker script syntax is very cryptic. Nor is it clear how you reference the included data from code.

One more approach I've found is the use of .incbin in assembly code, but I'm trying to avoid extra source code files, so I don't want to do that.

Finally, I wish I could just use the new C23 #embed, but according to here (https://en.cppreference.com/w/c/compiler_support/23) it's not supported by GCC yet. :(
Title: Re: Include a binary blob file into firmware executable at link-time
Post by: ejeffrey on August 12, 2023, 07:25:06 pm
Size should be bytes, the linker doesn't really know about C types.
Title: Re: Include a binary blob file into firmware executable at link-time
Post by: magic on August 12, 2023, 08:53:03 pm
IBut then you just get three symbols available for _binary_blob_bin_start, _binary_blob_bin_end, and _binary_blob_bin_size. I suppose I could just use the 'start' symbol as my array:

Code: [Select]
extern uint8_t _binary_blob_bin_start[];

But there are questions I have about this approach:

- What is the, uh, size of 'size'? uint32_t? Something else?
Theer is no _binary_blob_bin_size variable created at all. The value of the symbol is not a pointer to the blob's size stored somewhere, it's the size itself.
For instance, you could declare an extern variable of any type with this name, and simply take its address with &_binary_blob_bin_size to obtain the size.
(The variable wouldn't exist for real and attempts at accessing its value by C code could lead to memory faults or data corruption).

By the way, you don't need to create a temporary .o file, it is possible to link the binary blob directly into the final executable:
Code: [Select]
$ cat hello.txt
hello world
$ cat hello.c
extern char _binary_hello_txt_start[];

int main() {
        printf(_binary_hello_txt_start);
}
$ gcc -o hello -Wl,--format=binary hello.txt -Wl,--format=default hello.c
$ ./hello
hello world
Title: Re: Include a binary blob file into firmware executable at link-time
Post by: ejeffrey on August 13, 2023, 02:42:35 am
You can also ignore the size parameter and just use the start/end:

Code: [Select]
extern uint8_t _binary_blob_bin_start[];
extern uint8_t _binary_blob_bin_end[];

size_t blob_size;
int main()
{
   blob_size = (_binary_blob_bin_end - binary_blob_bin_start);
}

This is pretty annoying as you can't directly use sizeof on the symbol or otherwise use the symbol or its size at compile time. 

In C23 apparently you can #embed a binary file which is a bit ugly but more or less does exactly what I have wanted the language to directly support for 30 years.
Title: Re: Include a binary blob file into firmware executable at link-time
Post by: HwAoRrDk on August 13, 2023, 12:43:06 pm
Theer is no _binary_blob_bin_size variable created at all. The value of the symbol is not a pointer to the blob's size stored somewhere, it's the size itself.
For instance, you could declare an extern variable of any type with this name, and simply take its address with &_binary_blob_bin_size to obtain the size.
(The variable wouldn't exist for real and attempts at accessing its value by C code could lead to memory faults or data corruption).

I see. That's weird. So the address of the variable is the size itself. In that case I assume the size of the value will be uintptr_t, which is essentially the same as size_t.

By the way, you don't need to create a temporary .o file, it is possible to link the binary blob directly into the final executable:

Ah, cool. I didn't know you can do that.
Title: Re: Include a binary blob file into firmware executable at link-time
Post by: magic on August 14, 2023, 02:09:27 pm
By the way, you don't need to create a temporary .o file, it is possible to link the binary blob directly into the final executable:

Ah, cool. I didn't know you can do that.

A word of caution: this doesn't work with some file extensions (such as .c or .h or .cxx) because gcc assumes it's source code and tries to compile it instead of passing the file straight to the linker. Full list in man gcc. Not sure if this can be overriden?

edit
I suppose there will be no problem if one calls ld rather than gcc, but then all .c files must be compiled already into .o.