ARM GCC: How to convert a #included text file into C statements?

[peter-h]

I am creating a config file, using these statements

Code: [Select]

// Create default config file
  KDE_file_set_config_value ( "port1", "9600,8,n,1", "config.ini" );
  KDE_file_set_config_value ( "port2", "19200,8,n,1", "config.ini" );
  KDE_file_set_config_value ( "port3", "38400,8,n,1", "config.ini" );
  KDE_file_set_config_value ( "port4", "57600,8,n,1", "config.ini" );
which generates literally those strings in the file, in the form

port1=9600,8,n,1
port2=9600,8,n,1
etc

However, it is tacky to have to edit a .c file with this each time I want to change the default config. It is also not "safe" to open a project source file for editing. Hence I would like to be able to #include a text file containing the literal desired text

Code: [Select]

port1=9600,8,n,1
port2=9600,8,n,1etc

But how can the conversion be done?

Obviously one could generate an intermediate file by processing the text file with sed (which I have used for this kind of stuff) and then #include that file, (and have that sed processing as a "pre build step") but is there a way to do the processing in a .c source file, using some sort of macros?

IOW, how can I convert, at compile-time, a line in a #include file of

Code: [Select]

port1=9600,8,n,1
into

Code: [Select]

KDE_file_set_config_value ( "port1", "9600,8,n,1", "config.ini" );
The environment is ST Cube IDE, 32F417 target.

Many thanks for any tips

[ataradov]

You can't do exactly that, but you can make the file to contain lines like this:

Code: [Select]

DEFINE_PORT(port1, 9600, 8, n, 1)

Then before including that file in the code #define DEFINE_PORT to transform input arguments to your desired code.

[SiliconWizard]

If you don't insist on writing your default values in the same way they'll be written back as in the .ini files, then it becomes much easier.

The simplest way of course would be to define each value as a macro in the included text file:

Code: [Select]

#define CONFIG_VALUE_1     "port1", "9600,8,n,1"
#define CONFIG_VALUE_2     "port2", "19200,8,n,1"
...

And then include this file and just use values like so:

Code: [Select]

KDE_file_set_config_value (CONFIG_VALUE_1, "config.ini" );
KDE_file_set_config_value (CONFIG_VALUE_2, "config.ini" );
...

You could alternatively define an array in the include file, which would make code easier to maintain:

Code: [Select]

const struct { const char *key; const char *value; } DefaultConfigValues[]= {
    { "port1", "9600,8,n,1" },
    { "port2", "19200,8,n,1" },
...
};

and then in the code creating the default config file:

Code: [Select]

for (int i = 0; i < sizeof(DefaultConfigValues)/sizeof(DefaultConfigValues[0]); i++)
    KDE_file_set_config_value(DefaultConfigValues[i].key, DefaultConfigValues[i].value, "config.ini");

This way, adding config entries would be dead easy.

I don't see a way of using the entries exactly as you showed us in a text file and transform them into usable code without using some processing tool as you mentioned, and I think that would be much ado for not much. I find the above solution just as good. YMMV.

[retiredfeline]

I would have no qualms to doing it with a conversion step with sed or whatever instead of trying to work with the limited compiler preprocessor.

[Nominal Animal]

Can you show the exact definition of the KDE_file_set_config_value() macro?

[golden_labels]

The last solution, which SiliconWizard has suggested, is IMO the best one. Simply define a static const array of structures that describe the task to do.

If you really need to execute KDE_file_set_config_value with constant parameters, but know that I believe you do not, you may generate that using Boost.Preprocessor:

Code: [Select]

#include <boost/preprocessor/seq/for_each.hpp>
#include <boost/preprocessor/tuple/elem.hpp>
#include <whatever/is/needed/for/KDE_file_set_config_value.h>

#define CONFIG_PARAMS_ \
    (("port1", "9600,8,n,1"))\
    (("port2", "19200,8,n,1"))\
    (("port3", "38400,8,n,1"))\
    (("port4", "57600,8,n,1"))


#define CONFIG_PARAM_GEN_(z, aux, e)\
    KDE_file_set_config_value(BOOST_PP_TUPLE_ELEM(2, 0, e),\
          BOOST_PP_TUPLE_ELEM(2, 1, e), "config.ini");
static void generateKdeConfig(void) {
    BOOST_PP_SEQ_FOR_EACH(CONFIG_PARAM_GEN_, ~, CONFIG_PARAMS_)
}
#undef CONFIG_PARAM_GEN_
#undef CONFIG_PARAMS_
and then, in a file that is supposed to setup those values, simply include that header and invoke generateKdeConfig.

But, whenever possible, I would still tell you to follow SiliconWizard’s advice.

[ataradov]

Mine is way simpler and does not rely on boost, which may not be available on the target platform, and just ripping a couple things from it is not trivial usually.

Why make it so complicated?

Here is a complete example:

Code: [Select]

#include <stdio.h>

void KDE_file_set_config_value(char *a, char *b, char *c) // Just a demo implementation of this function
{
  printf("'%s', '%s', '%s'\n", a, b, c);
}

int main()
{
#define DEFINE_PORT(a, b, c, d, e)  KDE_file_set_config_value ( #a, #b "," #c "," #d "," #e, "config.ini" );

  //#include "your_config.txt"
  // This  will come from include
  DEFINE_PORT(port1, 9600, 8, n, 1)
  DEFINE_PORT(port1, 38400, 8, n, 1)
  DEFINE_PORT(port1, 115200, 8, n, 1)
  // End of the include file contents

  return 0;
}

The output of this code is

'port1', '9600,8,n,1', 'config.ini'
'port1', '38400,8,n,1', 'config.ini'
'port1', '115200,8,n,1', 'config.ini'

[peter-h]

Thank you all. I am amazed the compiler can't do the text processing

"Can you show the exact definition of the KDE_file_set_config_value() macro?"

It is a function. It creates the file (in the little 2MB FAT12 filesystem of my target) and appends the parameter=value to it, modifying the value if already existing. It's a bit like the windows .ini file format but simpler.

This question is regarding the neatest way to create a default file on a new target.

Later on the file will be editable with a web browser. I am told there is a simple way of implementing the parameter=value format, across HTML, but haven't got that far yet. The target has a web server, both HTTP and HTTPS.

[ataradov]

Why would compiler do text processing? Should it also have a differential equation solver, just in case you want to use that at some point? Use external tools if you need complicated processing.

And in this specific case what you can do with just the compiler is very close to what you want to do. I really don't see the issue here.

There are new languages coming up that have this capability, but this is so prone to abuse and I'm not sure how I feel about them.

[Kjelt]

Read up on the topic X-macros in c.
But be aware you're opening a potential can of worms 

https://en.wikipedia.org/wiki/X_Macro

[Nusa]

I'd just change the approach.
If config.ini doesn't exist or can't be parsed, copy default_config.ini into config.ini
If a required entry in config.ini doesn't exist, copy that entry from default_config.ini into config.ini
To update the defaults you just provide a new default_config.ini, which is way easier than compiling. Also doesn't require YOU to make the change in code.
If both files are missing, throw an error, cause someone deleted something they shouldn't have.

[Nominal Animal]

Can you show the exact definition of the KDE_file_set_config_value() macro?

It is a function. It creates the file (in the little 2MB FAT12 filesystem of my target) and appends the parameter=value to it, modifying the value if already existing. It's a bit like the windows .ini file format but simpler.

I do this by creating command-line utilities to use at build time (run on the build host), then use those utilities in the Makefile and/or shell scripts to generate/update the config files.  The interface would most likely be
    build-utils/set-config filename keyname value
noting that build-utils/ would be compiled for the host/build architecture and OS, not for the target arch/OS.

Even plain Makefiles allow conditional rules, so that if you use say make -DPORT1=9600,8,n,1 -DPORT2=38400,8,n,1 fsimage then build/utils/set-config ../fsimage/config.ini port1 9600,8,n,1 and build-utils/set-config ../fsimage/config.init port2 38400,8,n,1 would be run as part of building the filesystem image.

(This is just an example, of course.  I don't know the details, so I'm guessing at the build mechanism I'd implement myself.)

[peter-h]

Indeed, but once I create a "pre-build step" then I may as well have a default-config.ini file of the same format as the desired config.ini, and process it to an intermediate file with sed, and #include that intermediate file.

The advantage is that when the project is revisited in 5 years' time, it will be obvious what is what. I sell some products last built in 1995

I know how to do this, because I already have a "post-build step" configured which runs a little add-crc.exe utility (win32 command line) which appends a crc32 to the final binary.

I need to be careful about dependencies, but it should be ok because the pre-built sed process will always generate a fresh intermediate file, and the #include of that will make the .c file dependent on it - same as happens with any #include file - AIUI.

This will also need sed.exe to be in the project directory, because there are so many versions of these "unix" utilities kicking around. I've often been burnt a few times. I think cygwin is the usual source.

[SiliconWizard]

Note if you absolutely want your file format instead of one of the solutions we suggested, writing your own tool here would also be an option and shouldn't take more than maybe half an hour using your favorite language. I could do it in pure C in probably under 50 lines. Benefit is you wouldn't need to rely on any third party tool that may require updating, etc. Just make your tool a dependency of your project, and it will itself be built automatically before building the rest. Write it in 100% portable C, and you will never have any issue building it even 20 years from now. Just my 2 cents. Sometimes the time we take to find solutions largely exceeds the time it would have taken to take the plunge and do it yourself.

[peter-h]

Thank you all for your input.

I've decided to leave it like it was i.e. having the C statements in a #included file. Very simple!

If there were a lot more lines, the neater way might be to declare the text like this, in the #included file

Code: [Select]

uint8_t def_cfg [] = {
"port1", "9600,8,n,1", "config.ini",
"port2", "9600,8,n,1", "config.ini",
etc
};
and then have a bit of code stepping through the null-terminated strings and calling KDE_file_set_config_value() with each set of three.

[SiliconWizard]

It's basically what I suggested, except that you removed the structure. Maybe you find it easier to manipulate, because it requires fewer braces, but it'll make the code going through this array to call KDE_file_set_config_value() more complex. Your call of course entirely. I prefer structured data =)

Also, as you wrote it, it's not correct. You declared an array of uint8_t, whereas what it really is is an array of char *.

[peter-h]

Well, you know C very well, whereas I use it like I would use Fortran

I did some years ago, writing a bit of command line UI for a product, but my main C time is from 2021, and I stick to basics and very obvious code, with loads of comments. I do sort of understand structs (but don't like code which uses them for absolutely everything, which seems to be the fashion) but don't really get pointers (except when passing a function parameter by address, which is fine).

Isn't a char same as int8_t? Using uint8_t only affects doing arithmetic on the bytes, no?

[SiliconWizard]

Isn't a char same as int8_t? Using uint8_t only affects doing arithmetic on the bytes, no?

Uh, sort of. A char in C is not strictly supposed to be exactly 8 bits (yes that surprises many), although it's the most common case, whereas (u)int8_t types are.
But that's not the problem here. Your initializer is an array of string literals. String literals are not char's. They are compatible with pointers to char, or char arrays.

Here you should replace uint8_t with 'char *'. Ideally you should add const in front of that, otherwise it's likely to result in wasted RAM: the global variable will end up in RAM, initialized at startup with initializers stored in Flash. With a 'const' qualifier, no RAM should be used.

[Nominal Animal]

I much prefer SiliconWizard's example, but with just different whitespace and designated initializers to make it really easy to read:

Code: [Select]

const struct {
    const char *const file;
    const char *const name;
    const char *const value;
} config[] = {
    { .file = "config.ini", .name = "port1", .value = "9600,8,n,1" },
    { .file = "config.ini", .name = "port2", .value = "19200,8,n,1" },
    /* Explicit terminator entry */
    { NULL, NULL, NULL }
};
and the loop to handle these is just

Code: [Select]

    for (size_t i = 0; config[i].file != NULL; i++)
        KDE_file_set_config_value(config[i].name, config[i].value, config[i].file);

Compare to your own:

Code: [Select]

const char *const config[] = {
    "config.ini", "port1", "9600,8,n,1",
    "config.ini", "port2", "19200,8,n,1",
    NULL
};
with the loop being

Code: [Select]

    for (size_t i = 0; config[i] != NULL; i+=3)
        KDE_file_set_config_value(config[i+1], config[i+2], config[i+0]);
Considering long-term maintenance, I do believe bugs are far more likely with this latter form than with the former form.  Specifically, forgetting which index offset refers to which field.  (In these cases, I like to explicitly make the offset visible; therefore the file name is config[i+0] and not config, because the latter would differ from the other two, and possibly overlooked when a subsequent change was made.

The two should generate the same machine code, with the exception of the former config[] array having three and not just one NULL terminator pointers at end.

If these config snippets are made in different compilation unit (different source files) that are just linked together, then I would use ELF sections for it, but it does require ELF object file support from the toolchain.  It does not really matter what the final binaries are, as long as the intermediate object files are ELF format.
Then, you could put CONFIGSNIPPET(file,name,value); macros in file or function scope anywhere in the source files, and if that source file is linked to the same executable, those entries are automagically collected to a single array a simple for loop can process in the executable.