Loadable configuration files in C?

[ulix]

Hi everyone,

I would appreciate a bit of guidance for a project I'm currently working on.

My MCU (STM32F4) communicates with a slave device over SPI. The slave has a number of configuration registers that need to be initialized, and I would like to load register configurations from a file. At a later stage, I plan to add a W5500 Ethernet interface so that these register values can be read and modified remotely over the network.

My question is: What is the best way to store these register configurations in C? Should they be placed in a .c file or in a .h file? Also, would you recommend storing the configurations as, for example, uint8_t arrays, structures, or using #define macros? The reason I'm asking is that there is a development board for the device with accompanying software that can save the configuration of the device to a text file. I might use this output to generate a file that can be read by my MCU firmware. There are at least 20 registers to write to.

The configuration I'm leaning towards is as follows:

Code: [Select]

static const uint8_t reg_config[] = {
    0x00, 0x11, 0xAB, 0xFF, 0x00, 0x00,
};

Or would another approach be more appropriate?
I'm interested in hearing what is considered good practice for embedded projects.

Thanks!

[madires]

A "const uint8_t reg_config[]" is perfectly fine. If you have already some .h file with global variables then put it there. Otherwise place it where it's needed.

[pqass]

You could put the declaration in the .c but the array content in an include file where it can be generated (not necessarily hex) and/or versioned.

Code: [Select]

static const uint8_t reg_config[] = {
#include "config_a.csv"
};

[NorthGuy]

Could you clarify which of these do you want:

A. automatically replace the source code of your program, so you can later compile it and program to a new chip.

B. replace the variables in the HEX file (or other loadable) so that you can re-program your chip with a new HEX file without recompiling

C. receive the variables at run time and then re-write the flash in your MCU so that it always use new variables from now on

D. Externally supply new variables each time your program runs, so that it possibly uses different sets of variables every time it runs.

[Dave]

My go-to approach is to have a Python script that generates the corresponding parameters in a data structure that makes it easy to push the data onto the slave device at system initialization. How this structure looks depends mostly on how the data needs to be pushed to the slave device. The generated C source and header files are placed in a dedicated folder for auto-generated code, that is excluded from the Git repository. The script runs during the build process as part of the pre-build steps. This ensures that the data is always up to date.

If you want to optimize things, it is also nice to check the modification dates of the output files and compare that to the input file. If the input file is younger, you generate new output files, otherwise you end the script early.
This has two benefits:
1. the script finishes faster,
2. because the build tool checks the file dates in the exact same way to decide whether or not something needs to be recompiled, the auto-generated files will not be recompiled on each build run without modifying the input parameters.

[Pseudobyte]

If you move to C23 you can use the binary inclusion pre-processor directive.

Code: [Select]

#embed
https://en.cppreference.com/c/preprocessor/embed


Like this:

Code: [Select]

const uint8_t reg_config[] =
{
#embed "reg_config.bin" if_empty(0x00, 0x11, 0xAB, 0xFF, 0x00, 0x00)
};

//then get its size with
sizeof reg_config


[Fire Doger]

You could put the declaration in the .c but the array content in an include file where it can be generated (not necessarily hex) and/or versioned.

Code: [Select]

static const uint8_t reg_config[] = {
#include "config_a.csv"
};

I have seen this approach in industrial grade paid library. But it works only during build. (It's equal to opening the file with notepad and copy paste it's content before pressing build)

Since your target is to have these value runtime changeable, the only way to do it via storing them in a section in flash, and loading them to run before initializing the peripherals.

If you change the values over network, the mcu will save the new values in special section in flash and either:
A: reset
B: reinit the I2C peripherals with new values

Either way, during boot:
step_1: load from special flash section to ram structs
step_2: init I2C


Load and save to flash is whole can of worms. To store something in flash you need:
step_1: unlock flash
step_2: erase the page that you need to update (page size can be 512 byte, may be 4k bytes, depends on your mcu) (also erase = all become 0xFF)
step_3: write your data
step_4: lock flash

so you must allocate dedicated pages at the very end of the flash for storing parameters. Which is part of linker configuration.
"Very end" is not the only option, but you start going deeper in the can of worms... For first approach it's good and reliable. It allows you firmware to grow without overwriting preexisting configuration

Something that you definitely need to consider, is what will happen if the mcu reset during page erase or write and your stored data are corrupted.
There are 2 basic options, one is to load defaults (if it's acceptable) or allocate at least 2 pages at the end, and before erasing a page, create a copy of it. (Copy before update)

ex [1] [2] [3] [T]
if you need to update page 1, copy it to T(emp)
[1] [2] [3] [1]

Delete 1
[-] [2] [3] [1]

Write updated 1
[1'] [2] [3] [1]

Clear Temp
[1'] [2] [3] [T]

So at every step there is a backup of 1 before wrinting the new 1'. The only thing you need to do is then properly detect if page is corrupted and load back from the T(emp) at boot.

[ulix]

You could put the declaration in the .c but the array content in an include file where it can be generated (not necessarily hex) and/or versioned.

Code: [Select]

static const uint8_t reg_config[] = {
#include "config_a.csv"
};

I have seen this approach in industrial grade paid library. But it works only during build. (It's equal to opening the file with notepad and copy paste it's content before pressing build)

Since your target is to have these value runtime changeable, the only way to do it via storing them in a section in flash, and loading them to run before initializing the peripherals.

Maybe to keep it simple, I can "hardcode" the values as mentioend above. Any change over network, the register address is send and don't expect the register setting over ethernet is not retained.

[5U4GB]

The configuration I'm leaning towards is as follows:

Code: [Select]

static const uint8_t reg_config[] = {
    0x00, 0x11, 0xAB, 0xFF, 0x00, 0x00,
};


Depends how you want to set it up.  What I've done in a situation like this is:

Code: [Select]

#ifndef CONFIG_VALUES
  #define CONFIG_VALUES 0x00, 0x11, 0xAB, 0xFF, 0x00, 0x00
#endif
static const uint8_t reg_config[] = {
    CONFIG_VALUES
};

And then I can -DCONFIG_VALUES="0x12, 0x34 ..." when I build the code to override the defaults. That's for semi-static values known at compile time.

[Renate]

Oh, I've never heard of "embed". I think it's UGLY.
Just use a separate source file, put in a __attribute__ ((section(".whatever"))) if you care.

Is this config 8 bytes or 256 bytes?
If you have something configurable at build time you really should put a notation of that somewhere.
Maybe just a comment in the ELF file, maybe a text string, maybe something visible on the control panel.

Maybe a database (or even just a spreadsheet) and something that converts the data into source in your makefile.

[Pseudobyte]

... I think it's UGLY.

You're definitely entitled to that opinion. However some might argue that...

Maybe a database (or even just a spreadsheet) and something that converts the data into source in your makefile.

...is far more convoluted and obfuscated than a simple binary embedding. Unless you're forced to be on a specific version of C, I hardly see the point of not using #embed.

[SiliconWizard]

#embed just creates an array from a binary file, it's convenient, it's something people had been doing with ad-hoc solutions for decades.
Sure, if you already have your own ad-hoc solution that generates files for you, keep using that, but otherwise I don't see what would be "ugly" with #embed.

The only thing that can be annoying with #embed is file paths. If the source binary files are somewhere in your project hierarchy, just use relative paths. If not, using absolute paths would not look too good.

#embed is significantly faster in terms of compilation than a file generation tool + compiling the resulting file as C.

Note that GNU assembly files already had a .incbin directive long before C got #embed, so I was using that instead of generating C, and compilation, even for large binaries, was ultra fast. Note that you can generate "generic" assembly files for this, meaning target-agnostic, as they use no assembly code at all, just data.

[langwadt]

If you move to C23 you can use the binary inclusion pre-processor directive.

Code: [Select]

#embed
https://en.cppreference.com/c/preprocessor/embed


Like this:

Code: [Select]

const uint8_t reg_config[] =
{
#embed "reg_config.bin" if_empty(0x00, 0x11, 0xAB, 0xFF, 0x00, 0x00)
};

//then get its size with
sizeof reg_config


with gcc you can also use the assemblers .incbin

or you can use xxd -i  to turn a binary file in to a header file with an array