EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: DiTBho on January 26, 2023, 06:44:08 pm

Title: how to include a binary with C
Post by: DiTBho on January 26, 2023, 06:44:08 pm
How to include a binary into a C project?  :o :o :o

There are several ways ...

Today I had to include 8x16 fonts and 32x32 icons in an application that runs on ROM, and I thought it would be nice to post here how I did it :D

Code: [Select]
        .globl  suppa
        .size   suppa, 4
        .globl  suppa_size
        .size   suppa_size, 4

        .align 1
suppa:
        .byte  0xde,0xad,0xbe,0xaf
        .ascii "hAllo World!"
suppa_end:
        .align 4
suppa_size:
        .long  (suppa_end - suppa)
(assembly file, asm/data.s)

Code: [Select]
/* 
 * ... "public" is the same as "extern" in c89
 */

public uint8_t  suppa[];
public uint32_t suppa_size;

    my_size = sizeof(suppa);

    for (i0 = 0; i0 < my_size; i0++)
    {
        ... suppa[i0]
    }

    for (i0 = 0; i0 < suppa_size; i0++) /* that's how it works too, shouldn't it? */
    {
        ... suppa[i0]
    }
(c source, distilled example)

Code: [Select]
all: data ... 
        gcc  obj/data.o ... obj/app.o -o app

data: asm/data.s
        as asm/data.s -o obj/data.o
(Makefile, "as" is "GNU/AS")


To fill the asm/data.s file I used a tool that read the wanted binary file and outputs hex - byte per byte -
Title: Re: how to include a binary with C
Post by: bingo600 on January 26, 2023, 06:58:01 pm
I'd prob. just have read the binary , converted every byte into a 0x??, notation , and slapped a bin_data[] = {......} around the output.

/Bingo
Title: Re: how to include a binary with C
Post by: magic on January 26, 2023, 06:59:03 pm
To fill the asm/data.s file I used a tool that read the wanted binary file and outputs hex - byte per byte -
You aren't following the coolest thread on this subforum, are you? :box:
https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4637599/#msg4637599 (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4637599/#msg4637599)
Title: Re: how to include a binary with C
Post by: DC1MC on January 26, 2023, 07:06:26 pm
There are a lot of quick and friendly methods of linking binary files into a C program, I strongly recommend having a look here:
http://elm-chan.org/junk/32bit/binclude.html (http://elm-chan.org/junk/32bit/binclude.html)
Title: Re: how to include a binary with C
Post by: Siwastaja on January 26, 2023, 07:07:59 pm
Yeah, I used this method: https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4637629/#msg4637629 (https://www.eevblog.com/forum/programming/a-new-hardware-oriented-programming-language/msg4637629/#msg4637629) the very next day. A real timesaver compared to generating a .c file with an array, although it's not that much work. But I think the linking idea is more elegant, as well.
Title: Re: how to include a binary with C
Post by: langwadt on January 26, 2023, 07:18:17 pm
I'd prob. just have read the binary , converted every byte into a 0x??, notation , and slapped a bin_data[] = {......} around the output.

/Bingo

xxd -i  will do that for you

or use .incbin or the linker file to directly add the binary
Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 07:18:57 pm
I'd prob. just have read the binary , converted every byte into a 0x??, notation , and slapped a bin_data[] = {......} around the output.

Yup, sometimes it's even better to entirely stay with C files.

I like this other way because I can reuse some bin-file.s with assembly routines for hc11 without the need to look at other .C files, which would then have to be converted to assembly (uint8_t [..] into ".byte" field ....); I mean no problem, it's even script-able (python, lua ...), but it's just more work.

Crazy that I will recycle the same fonts with hc11 (isn't that too slow with a graphic LCD?  :o :o :o ) ... but that's another story ;D
Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 09:37:41 pm
xxd -i  will do that for you

Code: [Select]
overlay:dev-util/xxd

Code: [Select]
# echo "hAllo world" > hAllo.txt
# xxd -i hAllo.txt
unsigned char hAllo_txt[] =
{
  0x68, 0x41, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x0a
};
unsigned int hAllo_txt_len = 12;

nice tricks  :D
Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 10:03:20 pm
Code: [Select]
app-editors/vim-core-8.2.0360

updating vim-core (with USE=!minimal), adds /usr/bin/xxd
very nice  :D
Title: Re: how to include a binary with C
Post by: T3sl4co1l on January 26, 2023, 10:07:26 pm
I'm fond of the header format, too.

Rundown of options:
https://tratt.net/laurie/blog/2022/whats_the_most_portable_way_to_include_binary_blobs_in_an_executable.html
Binary can be linked directly as an object, it just needs to be named and tagged appropriately.  Although the commands shown here seem to be missing some things... not sure how the linker is supposed to know what to do with the object without a name, or can it default to its filename as the symbol?..  Anyway, you probably know more about linkers than I do, heh.

There's also an #embed (finally!):
https://thephd.dev/finally-embed-in-c23
I'd suggest updating your toolchain, but, uh... with the artisinal bespoke toolchain you have, I guess you can implement that at your convenience, or stick with the old ways?

Tim
Title: Re: how to include a binary with C
Post by: SiliconWizard on January 26, 2023, 10:17:11 pm
Didn't we go exactly over that a few days ago? ::)
Title: Re: how to include a binary with C
Post by: Sherlock Holmes on January 26, 2023, 10:39:23 pm
Didn't we go exactly over that a few days ago? ::)

Yes we did, where it was discussed how convenient it would be if we could initialize static entities directly from a raw file using some new compiler directive, some compile time executable operation.

Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 11:10:00 pm
Didn't we go exactly over that a few days ago? ::)

nobody mentioned the assembly method  ;D

and the guys here mentioned some pretty new ways, indeed I just found i needed to update my vim-core!!! and prepare a new overlay!!!

(thanks for that comment!)
Title: Re: how to include a binary with C
Post by: SiliconWizard on January 26, 2023, 11:13:36 pm
As I said in the other thread, to me, generating object files from binary files is the simplest, most straightforward way of doing it, and can be accessed from pretty much any language.
Doing that is dead-easy if you have binutils, but even if you don't, as long as you know the object file format for a given environment, writing the conversion tool is an easy task.
Just my 2 cents.
Title: Re: how to include a binary with C
Post by: brucehoult on January 26, 2023, 11:30:09 pm
It's pretty easy to convert a binary file to either an ASM file with .byte etc declarations, or a C .inc file with char or int etc values.

An advantage of a text format is that is may make life easier if you want to put it into a source code control system such as git.

Contrary to some opinion out there, git itself works just fine with binary files, and only storing deltas for them, etc. But many tools around git, such as "git diff" or the github web interface etc.
Title: Re: how to include a binary with C
Post by: Sherlock Holmes on January 26, 2023, 11:34:49 pm
As I said in the other thread, to me, generating object files from binary files is the simplest, most straightforward way of doing it, and can be accessed from pretty much any language.
Doing that is dead-easy if you have binutils, but even if you don't, as long as you know the object file format for a given environment, writing the conversion tool is an easy task.
Just my 2 cents.

How can that be simpler though? you need the compiler for the source code and another tool for data file, that's two distinct tools, or one must write a conversion tool. Surely being able to do that programmatically from within the source code is even easier? There are likely umpteen ways one can express the idea but having the compiler do it all for you has to be better - IMHO.








Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 11:52:12 pm
@SiliconWizard
Oh, well ... I have to do something similar with the DDE of my RISCOS v4.39 adjust  :o :o :o

There is no binutils(1) for RISCOS, there is no g-as, so ... you really like to only have { %.s , %.c , %.h } files.

DDE, what? It's the native development toolchain&ide of RISCOS. It's a commercial product and it's very powerful (you can even recompile some RISCOS modules with that) but it can appear limited if you try to compare with what you use for GNU/LINUX or *-BSD.

Why limited? Well, ... on DDE you have a C compiler(3) and an assembly (ARM-classic) compiler, and a linker. Plus other tools, which are for GUI and other specific RISCOS stuff. EndOfList. So, lot of tricks you use on GNU/Linux, *-BSD (including vim-core(2)) are not ... practical. Especially objdump and similar, and playing with native linker script file is not as flexible as you can do with GNU-gd.

So, you really like to have something you can select with DDE as "project files", and then click on "build it".

(1) actually, you can cross-compile { "make", "bash", "coreutils", "binutils", "gcc" } for RISCOS classic using a lib-unix compatibility layer on a special sandbox that handles files(5).

I did it, as well as I cross-compiled cross-compilers for RISCOS, and somehow - don't ask my how - that Frankenstein-stuff works, but it's very very slow and inconvenient, also due to the different filesystem.

In fact, on RISC files can't have an extension, so... if you have "hallo.c" you need to create a folder "c" and put "hallo" in that folder; thanks god, there is a (commercial) virtual filesystem module that you can invoke on the fly to adapt the RISCOS filename to the UNIX file name convention.

e.g.
unix app needs to operate with "hello.c" --> file-adapter  --> native RISCOS filesystem operates with "c/hello"
unix app needs to operate with "hello.c" --> file-adapter  --> native RISCOS filesystem operates with "h/hello"

Crazy? Insanely crazy if you consider that I paid 50 UKP for that (why?!?  "because science" :o :o :o ).

I think, "money well spent" if you consider that at least it also works over the network (4); just imagine how many things you have to change to get those UNIX-things to work.

That's why you don't want to use binutils tricks on RISCOS  :D

edit:
(2) vim, nano, joe ... this stuff needs an extra level of compatibility because there are no tty on RISCOS.
(3) there is also a C++ compiler, but it doesn't work, forget it!
(4) exported as NFS-v2, you no longer have to move files with a iomega ZIP/SCSI 50Mbyte cartridge.
(5) especially Bash needs to live into a sandbox in order to mimic stdin/out/err and redirection.
Title: Re: how to include a binary with C
Post by: DiTBho on January 26, 2023, 11:57:12 pm
An advantage of a test format is that is may make life easier if you want to put it into a source code control system such as git.

yup, to make life easier is *THE* reason we do that with Doors at work  :D
Doors is a commercial tool, I think the same concept applies to Git/SVN/...
Title: Re: how to include a binary with C
Post by: Sherlock Holmes on January 27, 2023, 12:06:07 am
@SiliconWizard
Oh, well ... I have to do something similar with the DDE of my RISCOS v4.39 adjust  :o :o :o

There is no binutils(1) for RISCOS, there is no g-as, so ... you really like to only have { %.s , %.c , %.h } files.

So consider writing a tool that can consume an arbitrary file, with some additional arguments and do as some have mentioned here, generate a C initialization source?

One could tell the tool that you're creating an array initializer for some kind of type:

Code: [Select]
convert_to_c <somepath> "int[4][3]" "arr" <Enter>
[/quote]

to get an output file that contains:

[code]
/* This text was generated by a tool on <date> at <time> */
/* do not modify this text.                              */

int arr[4][3] =
{
 { 2, 3, 1 },
 { 19, 12, 7 },
 { 10, 9, 8 },
 { 3, 11, 5 }
};

As some say out in these parts "Go get her writ!".

Or just

https://gist.github.com/albertz/1551304

https://github.com/TheLivingOne/bin2array/blob/master/bin2array.c











Title: Re: how to include a binary with C
Post by: Whales on January 27, 2023, 12:08:09 am
There's also an #embed (finally!):
https://thephd.dev/finally-embed-in-c23

Took me ages to find this link (was it #incbin, #binary or something?) only to discover you beat me to it. 

It's an interesting read.
Title: Re: how to include a binary with C
Post by: Sherlock Holmes on January 27, 2023, 12:14:21 am
There's also an #embed (finally!):
https://thephd.dev/finally-embed-in-c23

Took me ages to find this link (was it #incbin, #binary or something?) only to discover you beat me to it. 

It's an interesting read.

Well said, I agree with this myself:

Quote
You would think that we would have figured out a decent, cross-platform way to put data in executables after 60 years of filesystem research and advances and 40 years of C and C++ implementations.

Sound just like something I would write! anyway...

Quote
So much energy went into convincing people that yes, we should have this, no, we can’t “just parse better” our way out of it, no, the smartest people in the last 30 years across all major compilers were literally not capable of doing a better job at this! I actually wondered for a brief moment if I was not a proper user of C and C++ but an alien who existed in a planet that got the cursed or dark version of C and C++ and my interactions with other human beings were actually a brief window into an alternative universe where Everything Is Just Fine, Actually.

Buddy, I know how you feel, changing things is so much easier than changing attitudes.

Yes a very, very interesting read.


Title: Re: how to include a binary with C
Post by: DiTBho on January 27, 2023, 12:23:39 am
So consider writing a tool that

that works like xxd (now part of vim-core tools), just generating an assembly file like the one in my first post as further option.

xxd is already a great tool, with that option it would be more than great!

Why? Well, here ... I have to say, because with an assembly file you can easily tell the linker in which section you want it to put stuff.

.rodata?
.debug?
.custom?

no problem! no need to do weird stuff in C, no need to pass weird flags.
and Git will like it too!!!

It's what I am writing right now :D
Title: Re: how to include a binary with C
Post by: Sherlock Holmes on January 27, 2023, 12:27:19 am
So consider writing a tool that

that works like xxd (now part of vim-core tools), just generating an assembly file like the one in my first post as further option.

xxd is already a great tool, with that option it would be more than great!

Why? Well, here ... I have to say, because with an assembly file you can easily tell the linker in which section you want it to put stuff.

.rodata?
.debug?
.custom?

no problem! no need to do weird stuff in C, no need to pass weird flags.
and Git will like it too!!!

It's what I am writing right now :D

All I can say is that there's really nothing weird about using the same language to do both the code and the import of static data held in external files. Compilers read files for a living!

Read the article about #embed above.


Title: Re: how to include a binary with C
Post by: retiredfeline on January 27, 2023, 12:36:08 am
One tool I didn't see mentioned above, or in a link I didn't follow is https://srecord.sourceforge.net/ (https://srecord.sourceforge.net/). I use it for other purposes, but one of its capabilities is a C array declaration as an output format. The man page srec_cat details the relevant options.
Title: Re: how to include a binary with C
Post by: SiliconWizard on January 27, 2023, 02:42:12 am
@SiliconWizard
Oh, well ... I have to do something similar with the DDE of my RISCOS v4.39 adjust  :o :o :o

There is no binutils(1) for RISCOS, there is no g-as, so ... you really like to only have { %.s , %.c , %.h } files.

There must be a linker that can handle object files? If so just figure out the object file format and generate object files.
(Sure you can generate C files instead, your pick, but I like the option of generating object files directly - certainly more efficient for storing large binary blobs. And I'd be pretty intrigued if the object file format for your toolchain on RISCOS was not documented.)
Title: Re: how to include a binary with C
Post by: DiTBho on January 27, 2023, 10:01:37 am
There must be a linker that can handle object files? If so just figure out the object file format and generate object files.

yup, of course, but there is no tool to generate an obj, except the C89/99 and Assembly Compilers.

So, I would have to (port "objcopy"  to RISCOS and) play with something like this
Code: [Select]
objcopy \
        --prefix-symbol=_ \
        --input-target binary \
        --output-target elf \
         msg.txt msg.o
(ok, on RISCOS, it's not exactly "elf" for a native application, but ... ignore this for now)

That damned thing took 14 hours to compile inside a RISCOS-UNIX-sandbox (so, you can imagine how slow that Frankenstein stuff is), and then it even dared to complain :o :o :o

Code: [Select]
invalid bfd target

invalid, w_h_a_t_!?  :o :o :o :o

ah, that madness thing wants this
Code: [Select]
objcopy \
        --prefix-symbol=my \
        --input-target binary \
        --output-target elf32-little \
         msg.txt msg.o
(+        --output-target elf32-little)

This way it's happy, and outputs what it's needed.

Fine! Good! Excellent!

Code: [Select]
\()/
 __

let's start a whiskey flavored coffee celebration party :D :D :D

Just ... ummm ...

Code: [Select]
# objdump -t msg.o

msg.o:     file format elf32-little

SYMBOL TABLE:
00000000 l    d  .data  00000000 .data
00000000 g       .data  00000000 my_binary_msg_txt_start
0000000c g       .data  00000000 my_binary_msg_txt_end
0000000c g       *ABS*  00000000 my_binary_msg_txt_size

I am now on a MIPS32R2/LE, so "elf32-little" is ok here, but on my HPPA/BE it should be "elf32-big" instead.

Code: [Select]
# objdump -t msg.o

msg.o:     file format elf32-big

SYMBOL TABLE:
00000000 l    d  .data  00000000 .data
00000000 g       .data  00000000 my_binary_msg_txt_start
0000000c g       .data  00000000 my_binary_msg_txt_end
0000000c g       *ABS*  00000000 my_binary_msg_txt_size

And here I see the annoying part ... not, a real problem, if you export things as byte[..] there is no endian-problem at all, but ... ummm  .... that inconsistency (especially if automatically handled(1)) hits my eyes like a punch, and if I forget about it I'm sure that sooner or later a serious problem will lurk in there once I abuse it, then wasting ____a lot of time___ debugging :-//

* * *

So, moral of the story, I prefer "binary to C", or "binary to assembly", and to handle the result as a "module" filled with nothing but a public data described in its "interface".

          convert(binary) => module(interface, body(%.c | %.s))

This looks coherent, easy for Git, and portable with less details to care about.



(1) with cross-dev, that stupid thing caused a lot of problems with Gcc cross-emergent, not to mention a lot of problems with Portage cross-emergent. A stupid oversight with all the GNU-madness you need to look at (automake, autoconfigure, etc), thousands of hours wasted.
Title: Re: how to include a binary with C
Post by: DiTBho on January 27, 2023, 01:34:24 pm
Code: [Select]
rm -f obj/msg.*

objcopy \
        --prefix-symbol=my \
        --localize-hidden \
        --rename-section .data=.rdata,CONTENTS,ALLOC,LOAD,READONLY,DATA \
        --input-target binary \
        --binary-architecture powerpc \
        --output-target elf32-powerpc \
        msg.txt \
        obj/msg.o
(PowerPC/BE)

Code: [Select]
rm -f obj/msg.*

objcopy \
        --prefix-symbol=my \
        --localize-hidden \
        --rename-section .data=.rdata,CONTENTS,ALLOC,LOAD,READONLY,DATA \
        --input-target binary \
        --binary-architecture i386 \
        --output-target elf32-i386 \
        msg.txt \
        obj/msg.o
(x86/LE)

Code: [Select]
obj/msg.o:     file format elf32-powerpc

SYMBOL TABLE:
00000000 l    d  .rdata 00000000 .rdata
00000000 g       .rdata 00000000 my_binary_msg_txt_start
00000020 g       .rdata 00000000 my_binary_msg_txt_end
00000020 g       *ABS*  00000000 my_binary_msg_txt_size

Code: [Select]
public uint8_t my_binary_msg_txt_start[];
public uint8_t my_binary_msg_txt_end[];

private void app()
{
    p_uint8_t p_data;
    uint32_t  data_size;

    p_data    = my_binary_msg_txt_start;
    data_size = (my_binary_msg_txt_end - my_binary_msg_txt_start);

    printf("start=0x%p\n", my_binary_msg_txt_start);
    printf("  end=0x%p\n", my_binary_msg_txt_end);
    printf("#size=0x%lx\n", data_size);
    printf("#size=%lu\n", data_size);

    data_show(p_data, data_size, 8);
}

Code: [Select]
start=0x0x80052043
  end=0x0x80052063
#size=0x20
#size=32
|h A l l o   w |
|o r l d !   s |
|u p p a   c a |
|n   f l y   a |
|g a i n . . . |

so, it's arch/endian dependent  :o :o :o
Title: Re: how to include a binary with C
Post by: magic on January 27, 2023, 01:45:41 pm
What's endian dependent?

Nothing got mangled here.

The binary is embedded in its original byte order.
I suppose your data_show prints it the same way.
Title: Re: how to include a binary with C
Post by: DiTBho on January 27, 2023, 01:57:08 pm
What's endian dependent?

These lines

Code: [Select]
        --binary-architecture .... \
        --output-target .... \

architecture/endian dependent.
I mean, when you move the project among architectures, you have to "adapt" the configure file or something
(e.g. PowerPC64/BE (PowerMac-G5) is not the same as PowerPC64/LE (Cell,POWER9/10))

But see this:

Code: [Select]
objcopy \
        --prefix-symbol=my \
        --localize-hidden \
        --rename-section .data=.rdata,CONTENTS,ALLOC,LOAD,READONLY,DATA \
        --input-target binary \
        --binary-architecture i386 \
        --output-target elf32-little \
        msg.txt \
        obj/msg.o

Code: [Select]
- linking to my
/usr/lib/gcc/i686-pc-linux-gnu/10.2.0/../../../../i686-pc-linux-gnu/bin/ld: unknown architecture of input file `obj/msg.o' is incompatible with i386 output
collect2: error: ld returned 1 exit status
make: *** [Makefile:16: all] Error 1

objdump is happy with that
gcc/ld are not happy with that

(edit: with superhitachi and mips things get even more messed up
the target-flags have to match :o :o :o )
Title: Re: how to include a binary with C
Post by: magic on January 27, 2023, 02:06:44 pm
Yeah, object files are system dependent and it looks like you need to specify it explicitly for objcopy.

In the cool thread we used ld instead, which automatically generates its native object file. So command line is the same for all platform (as long as you use GNU ld).
Title: Re: how to include a binary with C
Post by: PlainName on January 30, 2023, 08:08:48 am
For anyone still wanting another choice, I'm currently using a slightly modified eight-year-old bin2c (https://sourceforge.net/projects/bin2c/) which has the advantage over the previously mentioned bin2array, and others, of processing a bunch of files all in one go and shoving them in a single .c/.h pair. My use at the moment is to embed a folder holding a bunch of HTML files into a RO virtual drive:
Code: [Select]
# Make HTML files into embedded binary source to place in RO memory.
file (GLOB_RECURSE bin_files RELATIVE ${CMAKE_SOURCE_DIR}/main/wifi/html ${CMAKE_SOURCE_DIR}/main/wifi/html/*)
add_custom_command(
OUTPUT ../main/wifi/html_files.h
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/main/wifi/html
COMMAND bin2c.exe -t -d html_files.h -o html_files.c ${bin_files}
COMMAND move html_files.* ..
DEPENDS ${CMAKE_SOURCE_DIR}/main/wifi/configurator.c)
add_custom_target(embed_html DEPENDS ${CMAKE_SOURCE_DIR}/main/wifi/html_files.h)
add_dependencies(${PROJECT_NAME}.elf embed_html)

Most of that is just getting bloody cmake/ninja to recognise a change and rung the thing again :( An advantage is I don't need to dick around editing file names in the build system or code.

The .h contents (for one of the files) ends up looking like this:
Code: [Select]
/* Contents of file framestyles.css */
extern const long int framestyles_css_size;
extern const unsigned char framestyles_css[1800];

and the relevant part in the .c:
Code: [Select]
/* Contents of file framestyles.css */
const long int framestyles_css_size = 1800;
const unsigned char framestyles_css[1800] = {
    0x2A, 0x20, 0x7B, 0x0D, 0x0A, 0x20, 0x20, 0x62 ...
};
Title: Re: how to include a binary with C
Post by: DiTBho on January 30, 2023, 10:33:21 am
the advantage over the previously mentioned bin2array, and others, of processing a bunch of files all in one go and shoving them in a single .c/.h pair. My use at the moment is to embed a folder holding a bunch of HTML files into a RO virtual drive

yeah, that's the right way to do things  ;D

Yesterday, I wrote my own tool

Code: [Select]
# make
- compiling app "mybin2cs" ... done
- finalizing to mybin2cs ... done

# echo "hAllo world" > msg0.txt
# mybin2cs msg0.txt obj/lib_suppa assembly.data suppa
# as obj/lib_suppa.s -o obj/lib_suppa.o
# objdump -t            obj/lib_suppa.o

obj/lib_suppa.o:     file format elf32-hppa
SYMBOL TABLE:
00000000 l    d  .text  00000000 .text
00000000 l    d  .data  00000000 .data
00000000 l    d  .bss   00000000 .bss
00000000 g       .data  00000000 suppa
00000010 g       .data  00000000 suppa_size

# cat obj/lib_suppa.s
        .data
        .globl suppa
        .globl suppa_size

        /* - - - body - - - */
        .align 1
suppa:
        .byte  0x68, 0x41, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f
        .byte  0x72, 0x6c, 0x64, 0x0a, 0x00, 0x00, 0x00, 0x00
        .align 4
suppa_size:
        .long 12

# cat obj/lib_suppa.h
#ifndef _lib_suppa_
#define _lib_suppa_

#include "types.h"

public uint8_t  suppa[];
public uint32_t suppa_size;

#endif

It always outputs a pair { .c, .h } | { .s, .h }.
If you specify "assembly", you can force the section you want { .data, .rodata, ... }.
Title: Re: how to include a binary with C
Post by: Nominal Animal on January 30, 2023, 10:36:42 am
I've used objcopy or ld, and a simple Bash script to generate both the ELF object file and the header file:
    #ifndef   NAME_H
    #define   NAME_H
    extern type name[count];
    #endif /* NAME_H
with count derived from the file size (file size divided by size of type); and NAME, name, type, the size of type, and the binary file name specified by command line parameters.

Then, sizeof name yields the size of the binary object in bytes, and (sizeof name / sizeof name[0]) == count yields the number of elements in the array, as usual.  The C/C++ compiler also detects trivial array overrun errors then, because it too knows the exact size of the array.

If you have several of them, say icons or images, I'd modify the script to read the list of binaries and their properties from a plain text file, generating one object file for all of them, with the associated header (and C source file) describing each of their properties.

Sure, one could easily write a program to do this, because ELF files are easier to manipulate than one would think, except for a few historical warts like 64-bit Alpha and S390, which use nonstandard fields, but because of Unix philosophy (https://en.wikipedia.org/wiki/Unix_philosophy), I prefer the scripted easily adapted approach myself.  (If you have lots of files, use the stat -c '%Y' filename utility to obtain the last modification time of the file in seconds since Unix epoch, and only regenerate the file if any of the sources is newer than the already existing object file.)



There is one pattern I wish everyone who writes sh-compatible scripts (including Bash scripts): Use an autodeleted temporary directory for your temporary files,
Code: [Select]
#!/bin/sh
export LANG=C LC_ALL=C
Work="$(/usr/bin/mktemp -d)" || exit 1
trap "rm -rf '$Work'" EXIT

# Use "$Work/filename" for temporary files, preferably with descriptive file names.
You want the script to work in a well-defined locale; the default (C) is the obvious choice.  This way, even if file names contain non-ASCII characters (using any character set), your script won't be confused, and just uses the file names as-is.

The temporary directory $Work is always removed when the script exits, even if the script fails due to an error.  The way the trap is set, the path to the temporary directory is expanded when the trap is set, so even if you manage to mangle the Work shell variable somehow (say, set it to /), the trap is not affected at all; the original temporary directory and all its contents will be removed.

(Both mktemp and the stat commands I described earlier are part of GNU coreutils, and always installed in Linux.  They or their equivalents are obviously available for other operating systems as well.)

Another pattern is to use nul-separated file names (find -print0, xargs -0, Bash read -d "", and so on), so that your scripts are not confused or do the wrong thing if a file name happens to contain a space or a newline.  But in build environments, that's usually not an issue.
Title: Re: how to include a binary with C
Post by: DC1MC on January 30, 2023, 12:00:53 pm
@Nominal Animal -> this is the  8) way !!!
Title: Re: how to include a binary with C
Post by: SiliconWizard on January 30, 2023, 07:38:38 pm
Now obviously if you don't have access to binutils and/or you don't want to write your own tool, sure, generating a C file is simple and one of the most portable approaches possible, while allowing to declare variables with the exact type you want (and alignment if required.) The only downside really is the size of the file - if you use hex as initializers you'll need in the order of 5 times the size of the binary file. But these days, storage is cheap and you can always delete the intermediate C file automatically after compiling it.
Title: Re: how to include a binary with C
Post by: brucehoult on January 30, 2023, 10:12:14 pm
The only downside really is the size of the file - if you use hex as initializers you'll need in the order of 5 times the size of the binary file. But these days, storage is cheap and you can always delete the intermediate C file automatically after compiling it.

I made a file with a million lines of random bytes in the format "0xnn,\n" [1] and then compressed it with default gzip, the same as git does. It came out 1.53x the size of a binary file of the same bytes.

Other options:

Code: [Select]
Size Time Method
6.00 0.00 raw
3.02 0.04 lz4
1.82 0.52 lz4 -9
1.53 0.42 gzip
1.48 0.07 zstd
1.43 5.09 gzip -9
1.02 0.39 bzip2

[1] perl -e 'for (1..1024**2){printf "0x%2x,\n",int(rand(256))}' >randhex_1M
Title: Re: how to include a binary with C
Post by: SiliconWizard on January 31, 2023, 12:31:43 am
Oh, if you compress, sure!

Otherwise, since it's usually just for generating a temporary C file that'll get compiled to an object file, you can write your conversion tool from binary to C to output on stdout, and pipe it to the C compiler. No intermediate file needed!