-
Hello,
I need some help / suggestions to do this in a better fashion. I would be very grateful for any suggestions and help.
The problem is this - I need to pass a structure member as a parameter to a function to set or clear some variables.
Presently, the best i could come up with is this
myfunction(mystruct_t *structure, char *key, char *value)
{
if (strncmp(desired_value_1, key, size) ==0)
{//do stuff}
if (strncmp(desired_value_2, key, size) ==0)
{//do other stuff}
if (strncmp(desired_value_1, key, size) ==0)
{//do yet anotherstuff}
}
As you can see, it is not very scalable nor elegant. I was wondering if it is possible to pass a "generic" struct member as an argument and just do the comparision against the structure passed in. Like so.
myfunction(mystruct_t *structure, struct_member *key, char *value)
{
if (strncmp(desired_value_1,structure->key, size) == 0)
{//do stuff}
if (strncmp(desired_value_2,structure->key, size) == 0)
{//do other stuff}
if (strncmp(desired_value_3,structure->key, size) == 0)
{//do yet anotherstuff}
}
obviously the type "struct_member" is just something I made up.. but it would allow the flexibility of reusing the logic to compare against any arbitrary data type. The second advantage I can think of is to ensure that it would obviously fail when the structure doesnt have the member defined in it. Third, it would allow me to use enumerations for th comparision, getting rid of unwieldy string operators and libraries.
Consequently, is there any way where I can simply use a char* to access a struct member, like so.
myfunction(mystruct_t *structure, char *key, char *value)
{
if (structure->key == 0)
{//do stuff}
if (structure->key == 2)
{//do other stuff}
if (structure->key == 99)
{//do yet anotherstuff}
}
I know I am sort of overreaching.. but I was hoping someone came up with a more elegant solution to this..
Thanks in advance!!
PS: Almost forgot to mention, this is for a microcontroller application, not a desktop one.
-
Linked list with TLV approach?
-
Linked list with TLV approach?
I am not sure what you mean.. I am not seeing how you can link linked lists :-P with a structure..
However, I am interested in learning more and I do not have an idea what TLV means ( A quick google says it stands for "Type Length Value" , but that is a reading for tonight..)
Doesnt it complicate it a lot?? Given the additional complexity, it might be easier to just have a switch cases and string comparision I suppose..
Also a limiting factor is my time to learn and implement.. While I am not on a deadline ( it is a hobby project for skill sharpening), I would like to
1. Keep it well documented.
2. Keep the code clean and readable / understandable.
Thank you for the suggestions anyway.. :-) If it is not too much trouble, can you provide an example? even pseudocode will do.. I just need to visualize the context you are suggesting in my head...
-
I'm not sure I fully understand what you actually need. There is no way to access a structure member by name stored in the variable, since that would need a run-time evaluation.
But at the same time I don't really understand why you would need something like this. Can you create a more complete example of what you need?
You can possibly minimize the amount of duplicate code with macros, but In many case it is not worth the added obscurity.
-
I'm not sure I fully understand what you actually need. There is no way to access a structure member by name stored in the variable, since that would need a run-time evaluation.
But at the same time I don't really understand why you would need something like this. Can you create a more complete example of what you need?
You can possibly minimize the amount of duplicate code with macros, but In many case it is not worth the added obscurity.
I am sorry, I should have provided a much more clearer example in my first post.
I have a a C struct, whose members I need to modify. I wanted to create a generic function to set any member of that struct. Like so
EXIT_STATUS SetPayloadKey(SensorPayload_t *payload, char *key, char *value);
Now, the structure itself has some ints, some floats, some char arrays..
And as I mentioned in the first post, It is doable by doing a string compare on the "key" and set / clear the members inside the logic.
However, it is limited to one type of struct. If I have multiple different structures, then I would need multiple such function each for a specific type of struct.
So I was wondering if it is in fact possible to have a generic function like so where the underlying logic is simply the equivalent of this.
SetPayloadKey(SensorPayload_t *payload, char *key, char *value){
payload -> key = value;
I can reuse this module for different types of structures.
I hope I made a little more sense :-).
It is a little lofty for generalizing across multiple structs.. but I am presently interested in just setting a member for one type of structure by passing the key directly.
Thank you! :-)
-
If "key" is a variable known only in run-time, then there is no way to do that. It is just too much logic compiler would have to generate. You would also need to know the size (type) of the field by its name.
It is possible to simplify the code from a complete copy-paste using macros, but it is really only worth it if you have more than 15-20 fields. Otherwise just do the manual thing.
-
What I've meant is to use linked lists instead of the structures. And each list element would contain a TLV set to manage your data. T(ype) would be the data type, L(ength) the data length and V(alue) the data itself. You could encode your structure members as type or add another variable as identifier, something like ID to identify the variable's name. This way you can simply search the linked list for an element with the ID of "name" and perform whatever check you like. The TLV part allows you to choose the right check, e.g. comparing strings or numbers. If linked lists are completely new to you then take your time.
-
If "key" is a variable known only in run-time, then there is no way to do that. It is just too much logic compiler would have to generate. You would also need to know the size (type) of the field by its name.
It is possible to simplify the code from a complete copy-paste using macros, but it is really only worth it if you have more than 15-20 fields. Otherwise just do the manual thing.
Unfortunately, it is known at runtime.. So you are right.. It is easier to do the check manually.
What I've meant is to use a linked list instead of the structure-. And each list element would contain a TLV set to manage your data. T(ype) would be the data type, L(ength) the data length and V(alue) the data itself. You could encode your structure members as type or add another variable as identifier, something like ID to identify the variable's name. This way you can simply search the linked list for an element with the ID of "name" and perform whatever check you like. The TLV part allows you to choose the right check, e.g. comparing strings or numbers. If linked lists are completely new to you then take your time.
With structures you could do some pointer magic but you need to know the data type of each member. If you group members of the same data type it would become a little bit easier, but it will be still quite ugly.
It is an elegant way on handling it! :-) Thank you. I am a greenhorn with linked lists.. I read about them.. but never had any use.. However, always a first time for everything..
If I am understanding you right.. this is what you are suggesting.
{
char *member_name;
int ID;
char *Type;
int Length;
char *Value;
linked_list *next_member;
} linked_list;
sorry.. I could only think of it as a C struct. :-P I can see where it adds some typechecking abilities.. But I will probably read up and try to see if I can implement it, provided my understanding of your suggestion is correct.
-
The linked list is a total overkill, and it will end up messier than just a simple bunch of if statements.
-
The linked list is a total overkill, and it will end up messier than just a simple bunch of if statements.
Oh yes definitely, for my application. I figured it might be a skill useful for later.. :-)
I have, for the record, about 7-8 members in any given struct. It is much much easier doing a simple lookup using strncmp.
-
Yep, the element of a linked list is a structure with data and a pointer to the next element. A variant is the doubly linked list with an additional pointer to the previous element. BTW, use typedef:
typedef struct element
{
uint8_t ID;
uint8_t Type;
uint8_t Length
void *Value
struct element *Next;
} element_type;
And yes, it's a pointer nightmare. ;D
-
The linked list is a total overkill, and it will end up messier than just a simple bunch of if statements.
I wouldn't recommend to use linked lists on 8 bit MCUs. ;)
-
Yep, the element of a linked list is a structure with data and a pointer to the next element. A variant is the doubly linked list with an additional pointer to the previous element. BTW, use typedef:
typedef struct element
{
uint8_t ID;
uint8_t Type;
uint8_t Length
void *Value
struct element *Next;
} element_type;
And yes, it's a pointer nightmare. ;D
Very elegant!! :-) Thank you.. It is much more clearer now.
As I have mentioned earlier.. it is an overkill for my application.. but it actually might be a more appropriate solution for a problem elsewhere!! :-D
-
So you want to use the struct, with the function call, as you would use an associative array in a high level language (HLL)?
For example, in JavaScript you can use:
foo["bar"] = baz;
which is equivalent to
foo.bar = baz;
and this will set the property bar to the value of baz.
In C, we need to construct all the data structures to hold these.
This might be a good lesson on the internals of HLLs --
On the one hand, it's a PITA to do in C. On the other, it may give you a great appreciation for all the hard work HLLs have to do, and why you might give them more patience for being slow, when they're so powerful. (and really, with the aggressive optimizations used in the most prominent languages, they're not very slow at all; an extremely impressive feat!)
One way JS might do this, is to store an "Object" as a structure in memory. The structure contains a type descriptor, so the interpreter can tell what it's looking at; and a pointer to the data thus described. The null object might have type 0 (and the data is irrelevant because it's never read), int 1, etc. Arrays, other Objects, and hybrids (mixed sets of objects, arrays or primitives) can be enumerated as well.
When we ask about properties of an object, we might be referring to an Object of, say, PropertyArray type, which contains key-value pairs. Say we do:
foo.one = "Apple";
foo.two = "Banana";
foo.three = "Cherry";
The interpreter might execute this and construct a PropertyArray whose data might be written in C like:
PropertyArray foo_p = {
struct {char* key = "one", char* value = "Apple"},
struct {char* key = "two", char* value = "Banana"},
struct {char* key = "three", char* value = "Cherry"}
}
When foo is accessed later,
console.log(foo.one);
foo.one = "Avocado";
a function is called which reads this PropertyArray, checking for the specified key, and returns or assigns a new value:
printf("%s", getPropertyValueString(&foo, "one");
setPropertyValue(&foo, "one", "Avocado");
...I take it you're interested in the latter operation, then?
We might break down the setting operation as getting a pointer to the PropertyArray's value member, and modifying it:
char** tmp = getPropertyValue_p(&foo, "one"); // found key; returns pointer to foo_p[0].value
mark_for_garbage_collection(*tmp);
*tmp = malloc(strlen("Avocado"));
strcpy(*tmp, "Avocado");
(Note we might be keen to mark the now-slightly-orphaned object for garbage collection. That is, to check if it has any other pointers to it in current context, and to free it if it's now completely orphaned. Garbage collection can also be done on its own, without such tips; it's up to the design of the VM. In C, we need to do this, explicitly, and carefully. Never free()ing the object results in a leak, while free()ing it unconditionally may leave a dangling pointer. Both are dangerous. So you can see, there is a lot of value added by automatic garbage collection.)
To further implement this, we need some way to store properties. If we are using dynamic memory (as in the HLL case), the PropertyArray can be anywhere in memory, and as we add and remove items from it, its size can vary; this is a normal dynamic array operation, the implementation of which can be found anywhere. We keep track of it with foo.
If we are allocating static types and properties, we might use a fixed array, and perhaps also split it into two arrays (char* keys[NUM_KEYVALUEPAIRS] and char* values[NUM_KEYVALUEPAIRS]) which can be a minor optimization. We might merge both into foo's type, so that it's a pair of fixed-length arrays. Or we might use a struct with useful member names, the same names being enumerated in the keys array, and the value array then gets offsets of foo's members; and perhaps their type or size/length as well, depending on what types and how general you want to get.
If the member assignments should have side-effects, you can also put together an array of function pointers; this reduces your stack of if-statements to a simple loop on strcmp, then performing the one operation on the element thus located.
A note on security: if any of these should be somehow user-writable, be very careful about buffer overruns, in static, dynamic (heap) or stack memory; and be especially careful of executing function pointers from RAM. A buffer overflow exploit is one thing; or jumping to a null pointer, another; but literally jumping to a potentially-user-controlled pointer is just putting it on a silver platter! If these data structures should be static, putting them in read-only memory (when an MMU is present) can help. (For global variables, set them as const, and any other attributes as necessary for the platform -- for example, AVR uses PROGMEM to place variables in Flash memory, and requires accessor functions to utilize. For functions, set them as static const, which will place them in global data rather than on the stack.)
Tim
-
The offsetof macro may be your friend here.
Completely untested...
struct foo {
int bar;
int baz;
};
void fn (void * s, uint32_t offset, int value){
int * ptr = (int*)((uint8_t*)s + offset);
*ptr = value;
}
int main () {
struct foo f;
fn (&f, offsetof (struct foo, bar), 5); // set f.bar to 5
fn (&f, offsetof (struct foo, baz),10); // set f.baz to 10
return 0;
}
There is another trick you can exploit, in that you could use the upper few bits in the member parameter to encode a type, so that you write the correct number of bytes. Obviously this relies upon all your structures having unused bits in the top and I would probably wrap the doings in a few macros to make things simpler....
Nice thing about this is that fn need have NO knowledge of the type of the structure, and in fact does not even need the structure declaration to be visible.
-
offsetof() will not help if you are getting the name of the member from outside of the program as a string.
-
True obviously, but it can be a reasonable thing to store in a table that defines how to set something when the string matches (And this way the file load and save functions do not need visibility of the metadata or the declaration of the structure they are reading or writing).
I am quite fond of static const tables of {name, type, offset, lower limit, upper limit, default value} for everything from parsing commands to loading and saving files, I likes my textual representations as load and save files, makes hand hacking them easy.
-
offsetof() will not help if you are getting the name of the member from outside of the program as a string.
It's not clear if OP is really getting the name of the member from outside the program. If that's not the case, simply removing the "" from the member's name will solve the biggest problem and possibly enable some kludge involving macros.
If OP really gets the names as text from outside the program, then the structures should be ditched and replaced with a key-value store. It doesn't need to be in the form of a list, this will work too:
struct kv {
char *key;
char *value;
int value_size;
};
struct kv kv_store_with_10_elements[10];
Hopefully you can imagine how to use that. This assumes that the external code treats the stored values as blobs of bytes and knows how to cast them to the intended types. If not, you could add another member to encode type of the stored value and write accessor function for every possible type... That would be easier in C++, probably.
-
@Teslacoil.
Yes, that is the lofty goal! :-) . However it looks like it is impractical for such a small application as mine.
However, that was an interesting read about how HLL operate and simplify some things. Thank you for that!
The offsetof macro may be your friend here.
Completely untested...
struct foo {
int bar;
int baz;
};
void fn (void * s, uint32_t offset, int value){
int * ptr = (int*)((uint8_t*)s + offset);
*ptr = value;
}
int main () {
struct foo f;
fn (&f, offsetof (struct foo, bar), 5); // set f.bar to 5
fn (&f, offsetof (struct foo, baz),10); // set f.baz to 10
return 0;
}
There is another trick you can exploit, in that you could use the upper few bits in the member parameter to encode a type, so that you write the correct number of bytes. Obviously this relies upon all your structures having unused bits in the top and I would probably wrap the doings in a few macros to make things simpler....
Nice thing about this is that fn need have NO knowledge of the type of the structure, and in fact does not even need the structure declaration to be visible.
Interesting!!! I was not aware of this macro.. looks like it miight provide an easier solution.. I ll try and mock something up with this. Thank you.
offsetof() will not help if you are getting the name of the member from outside of the program as a string.
True.. However, If I map the struct member names to a enumeration. Then I can pass the name as an enum literal to the function and translate that enum into an offset.
offsetof() will not help if you are getting the name of the member from outside of the program as a string.
It's not clear if OP is really getting the name of the member from outside the program. If that's not the case, simply removing the "" from the member's name will solve the biggest problem and possibly enable some kludge involving macros.
If OP really gets the names as text from outside the program, then the structures should be ditched and replaced with a key-value store. It doesn't need to be in the form of a list, this will work too:
struct kv {
char *key;
char *value;
int value_size;
};
struct kv kv_store_with_10_elements[10];
Hopefully you can imagine how to use that. This assumes that the external code treats the stored values as blobs of bytes and knows how to cast them to the intended types. If not, you could add another member to encode type of the stored value and write accessor function for every possible type... That would be easier in C++, probably.
Precisely my thoughts.. :-)
I do not really need to use a char* for a key name.. I did not / could not come up with a more clearer fashion to indicate the struct member.. However, using the offset macros, I might be able to pass an enum directly without using char* .
I dont get the names from outside the program.. But it passed during runtime.. For information.. It is running on a STM32F1 micro for a data logger application.
The key value stores is the end goal here however... This data will be finally transformed to a JSON string to be sent to a server.. It might actually be easier for me to use a key-value type datastructure right from the beginning..
Thank you for the ideas and suggestions.. :-)
-
I suspect you're dealing with device parameters/settings?
Then key-value list is the way to go internally.
struct listItem {
const char *name;
int key;
enum itemType type;
union {
void (*getInteger)(int key, int * const data);
void (*getFloat)(int key, float * const data);
...
} getters;
union {
void (*setInteger)(int key, const int * const data);
void (*setFloat)(int key, const float * const data);
...
} setters;
};
void getInteger(int key, int * const data){
find in list
check type
call getInteger (or getFloat if it was getFloat)
}
And either split over files or just in one bunch:
const struct listItem settings[] = { ... };
void getInteger_randomSettingsForThing(int key, int * const data){
*data = convertToMetricFromInternalUnit( yourstruct.member );
}
If someone want to set some key they can call setInteger(key, data)
and your top function looks in the list consisting of above entries if that is a valid option
and calls the specific function referenced in the item to deal with the action required when setting the value going with the key.
Where you actually store the value is irrelevant, this method eliminates global variables and adds options to do range checks or type conversions.
And it doesn't require casts so it's "type safe". Compared to passing everything via char pointers.
If it isn't for device parameters/settings then just ignore this.
-
I suspect you're dealing with device parameters/settings?
Then key-value list is the way to go internally.
struct listItem {
const char *name;
int key;
enum itemType type;
union {
void (*getInteger)(int key, int * const data);
void (*getFloat)(int key, float * const data);
...
} getters;
union {
void (*setInteger)(int key, const int * const data);
void (*setFloat)(int key, const float * const data);
...
} setters;
};
void getInteger(int key, int * const data){
find in list
check type
call getInteger (or getFloat if it was getFloat)
}
And either split over files or just in one bunch:
const struct listItem settings[] = { ... };
void getInteger_randomSettingsForThing(int key, int * const data){
*data = convertToMetricFromInternalUnit( yourstruct.member );
}
If someone want to set some key they can call setInteger(key, data)
and your top function looks in the list consisting of above entries if that is a valid option
and calls the specific function referenced in the item to deal with the action required when setting the value going with the key.
Where you actually store the value is irrelevant, this method eliminates global variables and adds options to do range checks or type conversions.
And it doesn't require casts so it's "type safe". Compared to passing everything via char pointers.
If it isn't for device parameters/settings then just ignore this.
It looks self contained more or less.. AFAICS. However wouldnt that require that you need one such structure for every different type of data structures you are trying to maintain? And If I understand it right.. the same unions for setters and getters can also be extended for a char* type or boolean type.
Thank you for the suggestions though! :-)
PS: Yes, it is for device parameters / settings.. However, It is not just for device parameters.. I am also looking to generalize it for Self Test results and the payload to be sent to a server, once data is ready..
-
For sake of completeness
This is one such data structure I have.
typedef struct GPS_Payload{
char* Device_Token;
float Latitude;
float Longitude;
float Altitude;
uint8_t Fix_Type;
} Payload;
and this is another
typedef struct Dev_Settings{
bool PMS5003_Enabled;
bool ECC508_Enabled;
bool ADS1015_Enabled;
char* Device_Token;
uint8_t Interval;
} SensorSettings;
I am basically, trying to make an easy job hard and come up with a way of making a generic function that can take pointers to (either) structures, structure members and values to set as arguments and operate accordingly.
-
Why would you even want a generic code to work with sturctures that have nothing in common with each other?
How will your generic code deal with being given different types of arguments at different times?
Suppose that structure->key really works in the example below,
myfunction(mystruct_t *structure, struct_member *key, char *value)
{
if (strncmp(desired_value_1,structure->key, size) == 0)
{//do stuff}
what's supposed to happen if structure->key isn't actually a text but a float?
Maybe you can simply write your function like that:
myfunction(char *key, char *value, size_t len, char *somethingelse) {
if (!strncmp(vaule, key, len)) {
blah blah do something with somethingelse
}
}
myfunction(&GPS_whatever.foo, "magic value", &GPS_whatever.bar);
It would probably take a bit of casting...
The thing is, C is not a dynamic language. It won't do typechecking at runtime for you. When you write code, you need to know which things will have which type in advance, or treat everything as char[] and then structures don't matter, just pass char* around.
-
Why would you even want a generic code to work with sturctures that have nothing in common with each other?
How will your generic code deal with being given different types of arguments at different times?
Suppose that structure->key really works in the example below,
myfunction(mystruct_t *structure, struct_member *key, char *value)
{
if (strncmp(desired_value_1,structure->key, size) == 0)
{//do stuff}
what's supposed to happen if structure->key isn't actually a text but a float?
Maybe you can simply write your function like that:
myfunction(char *key, char *value, size_t len, char *somethingelse) {
if (!strncmp(vaule, key, len)) {
blah blah do something with somethingelse
}
}
myfunction(&GPS_whatever.foo, "magic value", &GPS_whatever.bar);
It would probably take a bit of casting...
The thing is, C is not a dynamic language. It won't do typechecking at runtime for you. When you write code, you need to know which things will have which type in advance, or treat everything as char[] and then structures don't matter, just pass char* around.
Well, as of now it makes little differnce since the final goal is to convert them to json for sending to the server anyway.. So, a hacky workaround I have done is to define all the keys as char* ( I see that I have written this part wrong in the earlier post). All the keys are char*. That way the arguments are treated as such irrespective of the keys.. However, it would be nice to treat the data as its correct data type.
As you have mentioned.. it will be a problem with floats.. there is some sanitization / casting being done to ensure only a string is passed in as the "key" and the "value".
The thing is, C is not a dynamic language. It won't do typechecking at runtime for you. When you write code, you need to know which things will have which type in advance, or treat everything as char[] and then structures don't matter, just pass char* around.
I understand.. I am not trying to be cleverer than the people who have initially written the language nor the hundreds who have contributed to its development since.. :-) I have no doubt I ll fail miserably.. I am trying to make my code a little bit more robust and flexible at the same time. I know that sounds oxymoronic..
-
...
It looks self contained more or less.. AFAICS. However wouldnt that require that you need one such structure for every different type of data structures you are trying to maintain? And If I understand it right.. the same unions for setters and getters can also be extended for a char* type or boolean type.
Thank you for the suggestions though! :-)
PS: Yes, it is for device parameters / settings.. However, It is not just for device parameters.. I am also looking to generalize it for Self Test results and the payload to be sent to a server, once data is ready..
Yes, you would need one of those listItem entries per variable you need to access. Including getter/setter function. (the same thing you would need in c++)
Nothing limits you from referencing the same getter/setter function for multiple list entries, key is passed on. (you will need runtime key-type-location knowledge)
Yes, you can extend it to support char arrays, int arrays, or custom types. I have removed those from my example.
How you communicate with the above is a different layer. This part only provides a safe access method to any data via a public api.
Why would you even want a generic code to work with sturctures that have nothing in common with each other?
Code like this comes into play when you want to provide access to certain variables from your program from outside. In a layered and testable way.
For the above I wrote a metacompiler that assembles the items from troughout the code and creates one big consecutive list to link to the library.
But yes, true, you can get the same compiled result with a giant switch-case and a ton of global variables. However that is very manual and tightly coupled to layers tranceiving data.
Other things you may consider is ini files, json, or if you're brave, protobuf.
-
TLDR; Didn't fully read all the answers. I think, as some have probably suggested, what you are actually after is not a fixed struct, but some kind of dictionary (pairs of key/value), and there seems to be examples of that above. A dictionary can be implemented in various ways. Basically, that can be a table (array/list/whatever) of items, each being a struct containing at least the key and the value. To make the look-up faster, you can either sort the table by keys and do a binary search, or implement some kind of hash table. If the number of different keys is small, don't bother, a simple linear search should do the trick.
-
Why would you even want a generic code to work with sturctures that have nothing in common with each other?
I totally missed this... I can think of a couple of advantages of doing this.
1. Reuse of a common function block. You can have different data structure but one function to get or set them.
2. Code size.
3. Potentially eliminate errors in duplication.
4. Testability. You can simulate just one block of code with as many inputs as you want.
5. Extensibility. You can modify and change the data structures used elsewhere without changing the underlying logic.
-
TLDR; Didn't fully read all the answers. I think, as some have probably suggested, what you are actually after is not a fixed struct, but some kind of dictionary (pairs of key/value), and there seems to be examples of that above. A dictionary can be implemented in various ways. Basically, that can be a table (array/list/whatever) of items, each being a struct containing at least the key and the value. To make the look-up faster, you can either sort the table by keys and do a binary search, or implement some kind of hash table. If the number of different keys is small, don't bother, a simple linear search should do the trick.
Thank you.. It would appear that is what I am looking for.. I ll work on the suggestions above and post what I can come up with.. and put it up for scrutiny! :-)
-
TLDR; Didn't fully read all the answers. I think, as some have probably suggested, what you are actually after is not a fixed struct, but some kind of dictionary (pairs of key/value), and there seems to be examples of that above. A dictionary can be implemented in various ways. Basically, that can be a table (array/list/whatever) of items, each being a struct containing at least the key and the value. To make the look-up faster, you can either sort the table by keys and do a binary search, or implement some kind of hash table. If the number of different keys is small, don't bother, a simple linear search should do the trick.
Thank you.. It would appear that is what I am looking for.. I ll work on the suggestions above and post what I can come up with.. and put it up for scrutiny! :-)
That may be, but reading your post just above, I'm not completely sure this is really what you want.
Looks like you want this dictionary approach from a software dev POV, but not necessarily as a feature within the software itself.
One obvious drawback of a dictionary is that accessing "members" will always be slower than directly accessing a struct member in C. You may then have performance issues in your software if it matters (it may not in a given situation, so it's up to you.) Also, if members can have different types, you'll need to store the corresponding "type" along with each "key", and the corresponding data content. And the accessor functions will need to deal with this. Drawback beside complexity, is that you won't get the static checks a compiler can do regarding type compatibility and such.
A mixed approach is possible (I've used it to implement easier-to-maintain file read/write and UI (value modification) on top of data structs.)
You define your structs in a normal way, but create additional dictionaries that refer to the members of the structs. The way you can do that is with pointers, and the offsetof() operator. You need to be very familiar with pointers though. Each dictionary entry would be a key, and a reference to the member of a struct (in the form of its offset from the start of the struct, that you get with offsetof().) You'd also need to define a few macros to help writing the necessary code without too much duplication. Then you can access struct members either by "key", or directly. All of this (pointer arithmetic, offsetof(), macros...) can be a bit frowned upon by some though. Up to you.
-
Okay.. I have had some time to read, understand and assimilate the suggestions given above.. and I though I will start off with the simplest solution, for now.
The offsets seem like a pretty good solution to achieve most of what I am looking for.
So.. I tried to see if I can come up with how to make a sort of generic function using offsets, and this is what I could think of.
typedef struct foo
{
uint8_t alpha;
uint8_t bravo;
uint8_t charlie;
} foo_t;
enum foo_t_keys { alpha = offsetof(foo_t, alpha), bravo = offsetof(foo_t, bravo), charlie = offsetof(foo_t, charlie) };
typedef struct foo2
{
uint8_t delta;
uint8_t echo;
uint8_t foxtrot;
} foo2_t;
enum foo2_t_keys { delta = offsetof(foo2_t, delta), echo = offsetof(foo2_t, echo), foxtrot = offsetof(foo2_t, foxtrot) };
void setter (void *s, uint32_t offset, int value)
{
int *ptr = (int *)((uint8_t *)s + offset);
*ptr = value;
}
int main()
{
foo_t foo;
foo2_t foo2;
setter(&foo, alpha, 100);
setter(&foo, bravo, 20);
setter(&foo, charlie, 30);
setter(&foo2, delta, 250);
setter(&foo2, echo, 200);
setter(&foo2, foxtrot, 150);
printf("Value of foo.alpha is %d\n", foo.alpha);
printf("Value of foo.bravo is %d\n", foo.bravo);
printf("Value of foo.charlie is %d\n", foo.charlie);
printf("Value of foo2.delta is %d\n", foo2.delta);
printf("Value of foo2.echo is %d\n", foo2.echo);
printf("Value of foo2.foxtrot is %d\n", foo2.foxtrot);
return 0;
}
Basically have a standard C structure and an enum mapping the keys to the offsets. This only works if the enum is in sync with the structure.
and have a generic setter function that takes a structure (void*) , offset (uint32_t) and value (uint8_t) as the arguments.
The above example compiles fine and works.
One huge caveat is that it only works if the structure is made up of the same kind of data type members. I am still thinking of how to modify the setter accept any kind of data type for the value and cast it internally using a lookup..
Still a WIP for the different types of values.. I might need to have another enum(in sync with the structure) tracking the data types and have some sort of lookup to cast the argument structure and the offset... I m guessing this might require some macro magic...
-
I don't get it. If the name of the member is not a string and known at compile time (like in the code above), then a simple macro will work:
#define setter(s, member, value) (s)->member = value
or depending on the way struct is passed
#define setter(s, member, value) (s).member = value
But I don't see how is this helpful at all.
-
I don't get it. If the name of the member is not a string and known at compile time (like in the code above), then a simple macro will work:
#define setter(s, member, value) (s)->member = value
or depending on the way struct is passed
#define setter(s, member, value) (s).member = value
But I don't see how is this helpful at all.
You are indeed right.. It works.. and even for strings.. Looks like I wasted mine and a lot of fourm members time for very little returns. I apologize for this.
The members are known at compile time. I think simply maintaining a enum in sync with the actual struct members should solve what I was looking for in the first place.
Thank you @ataradov. This is indeed simple and valuable! :-)
PS: A similar macro works even for a getter function.. for strings.. bools.. ints.. as long as the struct in question has the member defined.. the macro will work.. if not.. then it throws a compilation error.
-
The general idea of being able to take a structure definition such as
struct foo {
sometype_1_t bar;
sometype_2_t baz;
sometype_3_t frob;
sometype_4_t fizzle;
};
and be able, at runtime, to get info about the name and type of each element and to access the elements using a string containing the field name falls under the general heading of "type introspection" (and "reflection" for actual access to the variables rather than just type info).
It can be useful if you want to write generic code to, say, pick up configuration parameters from a text file but have regular data structures in your program rather than key-value dictionaries.
Some languages have introspection and reflection built in, but not C, you can do something manual and play preprocessor games to help build the necessary data structures to make it work and perhaps even somewhat type-safe but it will always be a question of "rolling your own".
-
I did that once, but quickly dropped it due to the risks (wrong type, unaligned access, and other manual errors)
Looked like this:
struct item {
void* base;
uint16_t offset;
enum type;
}
#define item_init(base, element, type) { &base, offsetof(base, element), type }
You can always make a metacompiler, like qmake, that creates such list.
For example using: https://pypi.org/project/pycparser/
-
The general idea of being able to take a structure definition such as
struct foo {
sometype_1_t bar;
sometype_2_t baz;
sometype_3_t frob;
sometype_4_t fizzle;
};
and be able, at runtime, to get info about the name and type of each element and to access the elements using a string containing the field name falls under the general heading of "type introspection" (and "reflection" for actual access to the variables rather than just type info).
It can be useful if you want to write generic code to, say, pick up configuration parameters from a text file but have regular data structures in your program rather than key-value dictionaries.
Some languages have introspection and reflection built in, but not C, you can do something manual and play preprocessor games to help build the necessary data structures to make it work and perhaps even somewhat type-safe but it will always be a question of "rolling your own".
That is the sort of use case I am referring to.. an datalogger that powers up.. reads a configuration file.. acquires/ processess the data, sends to server.. rewrites the configuration parameters to a file and sleeps..
While it sounds convoluted, about the reading and rewriting the configuration parameters back, it has a good reason. The self test performed at the beginning of every cycle store the results during the iteration lifetime in memory.. and any sensors that fail the ST are written as such to the configuration file to a non volatile storage.. On next boot, they would not even be attempted since they would / should have been disabled in the previous write to file.
However, it is much much easier to read a config file into a a traditional C structure and do any operations on them instead of constantly looking up a key value in a dictionary or file.. hence the first step of reading the configuration file, parsing the contents and constructing a structure to hold the parsed values.. It is here that the runtime construction of a structure and its associated type, length considerations came into play.
-
In Linux, one can use ELF sections to do the heavy lifting:
#include <stdlib.h>
#include <stdint.h>
#include "conf.h"
struct foo {
int x;
int y;
float z;
};
/* Generic structure operations, use when declaring structures of this type */
CONFIG_OPS(struct_foo_ops) = {
CONFIG_OP("x", offsetof(struct foo, x), CONFIG_TYPE_INT),
CONFIG_OP("y", offsetof(struct foo, y), CONFIG_TYPE_INT),
CONFIG_OP("z", offsetof(struct foo, z), CONFIG_TYPE_FLOAT),
CONFIG_OPS_END
};
/* An instance of above configurable structure */
struct foo bar;
CONFIG_STRUCT("bar", &bar, struct_foo_ops);
/* Another instance of above configurable structure */
struct foo baz;
CONFIG_STRUCT("baz", &baz, struct_foo_ops);
so that at runtime,
CONFIG_FOREACH(st) {
printf("Structure \"%s\" at %p has:\n", st->name, st->base);
CONFIG_FOREACH_OP(op, st) {
printf(" Member \"%s\" at %p (type %d)\n", op->name, (void *)(op->addr + (uintptr_t)st->base), op->type);
}
}
prints
Structure "baz" at 0x55c7b19f0068 has:
Member "x" at 0x55c7b19f0068 (type 1)
Member "y" at 0x55c7b19f006c (type 1)
Member "z" at 0x55c7b19f0070 (type 2)
Structure "bar" at 0x55c7b19f0078 has:
Member "x" at 0x55c7b19f0078 (type 1)
Member "y" at 0x55c7b19f007c (type 1)
Member "z" at 0x55c7b19f0080 (type 2)
but I've never tried to use this on a microcontroller. I do believe it works using avr-gcc and arm-gcc on Linux (that is, compiling on any Linux machine to AVR or ARM targets using GCC), just haven't done it. (The key is that one must compile an ELF binary for the target, you see. This facility is an integral part of the ELF file format.)
Essentially, each CONFIG_OPS(varname) defines an array of struct config_op's,
#define CONFIG_TYPE_INT 1
#define CONFIG_TYPE_FLOAT 2
struct config_op {
const unsigned char *name;
uintptr_t addr;
uint_fast8_t type;
};
and each CONFIG_STRUCT(strname, addr, ops) declares a static const variable (with a name generated based on the line number) emitted to a special section; I used config_ops:
struct config_struct {
const unsigned char *name;
void *base;
const struct config_op *ops;
void *padding;
};
The structure size does need to have size a multiple of section alignment (you can adjust that using a linker script; I just padded the structure). GCC helpfully provides external symbols __start_config_ops and __stop_config_ops (as long as the section name is suitable for a C variable name), whose addresses (&__start_config_ops and &__stop_config_ops) span the variables emitted to this section in all compilation units.
The macro-shenanigans are fairly ugly, and you do need to link the object files to an executable, but this mechanism is useful for link-time merging of entries into a single contiguous array. It's used fairly often in the Linux kernel, for example. Also, the ELF constructors/destructors (executed before and after main()) use a similar mechanism, with .init_array and .fini_array sections containing the addresses of the functions to be executed.
The annoying part of this kind of scheme is still to write the accessor functions; i.e., conversion to and from string, when given a pointer to the target variable and its type number.
Overall, the ROM overhead is two pointers and one/two integers per member in a modifiable structure type, plus two pointers and one/two integers for the terminating marker. The second integer is for a hash (I recommend DJB2 xor variant), to speed up string lookups. Each modifiable structure of any type requires three pointers plus an optional integer, for hashing the name. Plus the string literals for all the names.
So, in general, this approach would work very well if you had many instances of the same type of structures that can be accessed via their names. (Parsing dot-separated names like foo.x to locate the target member is easy.)
-
For sake of completeness
This is one such data structure I have.
typedef struct GPS_Payload{
char* Device_Token;
float Latitude;
float Longitude;
float Altitude;
uint8_t Fix_Type;
} Payload;
and this is another
typedef struct Dev_Settings{
bool PMS5003_Enabled;
bool ECC508_Enabled;
bool ADS1015_Enabled;
char* Device_Token;
uint8_t Interval;
} SensorSettings;
I am basically, trying to make an easy job hard and come up with a way of making a generic function that can take pointers to (either) structures, structure members and values to set as arguments and operate accordingly.
your question is still unclear... the only common thing between Payload and SensorSettings is member Device_Token. so i assume you want to operate/modify this member in a generic function? you are asking for disaster esp the way you arrange the members order like that. what you seek probably is class (or structure) inheritance or polymorphism in OOP.
btw: stupidity is the lack of intelligence. each goes in the other direction of the other. both if summed equal to 100%... 46% intelligence means there is 54% stupidity left. 100% intelligence means 0% stupidity left. so if the 100% intelligence matter made something from His image, so we can safely assume the image is only has 1% intelligence, so guess what the 99% is? cheers.
-
The usual way of parsing config files is to simply hand write every option. It's known at compile time, after all. This can be streamlined into matching lines to strings (from an array) and getting pointers/offsets and types from there. Which ends up very similar to what we have above, so that checks out.
The simpler, MCU-resident way to do what you describe, would be to just store some flags in EEPROM and check them before testing the respective sensor.
Tim
-
Interesting ideas! But who wants to maintain source code with so much complexity?
-
For sake of completeness
This is one such data structure I have.
typedef struct GPS_Payload{
char* Device_Token;
float Latitude;
float Longitude;
float Altitude;
uint8_t Fix_Type;
} Payload;
and this is another
typedef struct Dev_Settings{
bool PMS5003_Enabled;
bool ECC508_Enabled;
bool ADS1015_Enabled;
char* Device_Token;
uint8_t Interval;
} SensorSettings;
I am basically, trying to make an easy job hard and come up with a way of making a generic function that can take pointers to (either) structures, structure members and values to set as arguments and operate accordingly.
your question is still unclear... the only common thing between Payload and SensorSettings is member Device_Token. so i assume you want to operate/modify this member in a generic function? you are asking for disaster esp the way you arrange the members order like that. what you seek probably is class inheritance or polymorphism in OOP.
btw: stupidity is the lack of intelligence. each goes in the other direction of the other. both if summed equal to 100%... 46% intelligence means there is 54% stupidity left. 100% intelligence means 0% stupidity left. so if the 100% intelligence matter made something from His image, so we can safely assume the image is only has 1% intelligence, so guess what the 99% is? cheers.
For now I am good. ataradovs macros did the job!! :-)
You are absolutely right in that I am looking at polymorphism type of programming in C.
I gave those structures simply because they were the most easiest to show. I ll try and explain what I set out to do.. I have a bunch of structures, with different members and different member types. I want to have a generic setter function that takes a structure, structure member and value and then sets the specified member to the passed in value. similarly a getter function. And as it turns out.. I was overcomplicating and a string comparision would do the job just fine and probably more easy to implement.
btw: stupidity is the lack of intelligence. each goes in the other direction of the other. both if summed equal to 100%... 46% intelligence means there is 54% stupidity left. 100% intelligence means 0% stupidity left. so if the 100% intelligence matter made something from His image, so we can safely assume the image is only has 1% intelligence, so guess what the 99% is? cheers.
It was meant as a wry attempt at humour.. not intended to hurt the sentiments of anyone..
The usual way of parsing config files is to simply hand write every option. It's known at compile time, after all. This can be streamlined into matching lines to strings (from an array) and getting pointers/offsets and types from there. Which ends up very similar to what we have above, so that checks out.
The simpler, MCU-resident way to do what you describe, would be to just store some flags in EEPROM and check them before testing the respective sensor.
Tim
Yes that is how I started. An external flash is used to store some device logs.. so made sense to move the settings offboard from the MCU as well. Hence the entire exercise..
Interesting ideas! But who wants to maintain source code with so much complexity?
Interesting indeed. I have to I suppose... since it is a side project of mine.. :-P
-
For now I am good. ataradovs macros did the job!! :-)
You are absolutely right in that I am looking at polymorphism type of programming in C.
I gave those structures simply because they were the most easiest to show. I ll try and explain what I set out to do.. I have a bunch of structures, with different members and different member types. I want to have a generic setter function that takes a structure, structure member and value and then sets the specified member to the passed in value. similarly a getter function. And as it turns out.. I was overcomplicating and a string comparision would do the job just fine and probably more easy to implement.
or probably "function template" feature in C++. with string comparison, you'll have to expand/modify your generic function (setter or getter regardless) as your number of structures is expanding. with built in language features, all your comparisons, virtual tables and function expansions is taken cared of behind the curtain, just dont expect fancy codes can run accordingly in nonfancy machines however you try to squeeze it.
-
Interesting ideas! But who wants to maintain source code with so much complexity?
If you need it, you need it, so no choice but to maintain it.
But I'd rather maintain a decent hand-rolled introspection/reflection implementation which keeps the complexity in one place (and, if done well, doesn't need to be touched again) than a rag-tag set of enums accessor macros and stuff which needs to be kept carefully in sync with the structure definitions.
-
you are asking for disaster esp the way you arrange the members order like that. .
Would you mind elaborating on that? Is there something wrong with the way my structures are laid out?
I know it is not the most elegant, but I do not see something obviously wrong with it.
-
Keep in mind that if access performance is not a key issue, you could still implement some kind of dictionary and do without C structs altogether, which would require less messing around. Also, there are multiple libraries for that out there. That would basically be a mini-database.
-
I have two takes...
Option 1
Linked list, maybe with hashing to speed up accces
struct key_value_pair {
struct key_value_pair *next;
char *key;
char *value;
};
and in the top level structure...
struct structure {
struct key_value_pair *first_keyvalue;
... other members ...
}
Good points - Can have as many keys and values as you want, with values of unlimited size (well, as much as memory allows)
Bad points - Need to use dynamic memory allocation, so memory leaks are possible.
Option 2
If you know the maximum count and sizes of key+value pairs you might have, then...
struct key_value_pair {
char key[MAX_KEY_LEN+1];
char value[MAX_VALUE_LEN+1];
};
struct structure {
struct key_value_pair key_value_pairs[MAX_KEY_VALUE_PAIRS];
... other members ...
};
Pros: Avoids dynamic memory allocation.
Cons: All memory for the maximum case needs to be allocated up front. Hard to speed up.
In the latter case adding, searching, updating are relatively simple, with the only error cases of 'key' or 'value' being too long, or if MAX_KEY_VALUE_PAIRS is exceeded.
If you code your own linked lists there are a few different corner cases to deal with (performing operations on an empty list, performing operations on the head of the list, performing operations in the body of the list, performing operations on the tail), along with the chance of running out of memory or memory fragmentation. You will write buggy list code until you get the pattern fixed in your head.
-
Keep in mind that if access performance is not a key issue, you could still implement some kind of dictionary and do without C structs altogether, which would require less messing around. Also, there are multiple libraries for that out there. That would basically be a mini-database.
It is going to run on a STM32F1 micro.. I am guessing it is going to be of some concern. The overall firmware has very little timing constraints. I would prefer to stick to a C struct primarily for its simplicity.
I have two takes...
Option 1
Linked list, maybe with hashing to speed up accces
struct key_value_pair {
struct key_value_pair *next;
char *key;
char *value;
};
and in the top level structure...
struct structure {
struct key_value_pair *first_keyvalue;
... other members ...
}
Option 2
If you know the maximum count and sizes of key+value pairs you might have, then...
struct key_value_pair {
char key[MAX_KEY_LEN+1];
char value[MAX_VALUE_LEN+1];
};
struct structure {
struct key_value_pair key_value_pairs[MAX_KEY_VALUE_PAIRS];
... other members ...
};
The first option looks really straightforward. However, as I have mentioned earlier, ataradovs macros sort of addressed the requirement.
Thank you @hamzter_nz and @siliconwizard for your suggestions. :-)
One a seperate note, ataradovs answer provided with a basic set of macros to do the job. I ran into some other issues while trying to mock / unit test the macros.. but as a work around I settled on verifying that the struct member had a changed value pre and post the macro invocation rather than mock the macro itself. That is a seperate thread in and of itself.
So what I am trying to saying thanks a lot fellows!! :-) It has been a very educative / productive discussion with plenty of helpful suggestions.
-
You know, if all your configurable items are global variables, or items in global structures, you can use
struct config_entry {
const char *const name; /* In Flash/ROM */
void *const addr; /* Address of variable in RAM */
const uint32_t hash:24; /* 24-bit hash */
const uint32_t type:8; /* 8-bit type */
};
/* Example types */
#define TYPE_BIT0 0
#define TYPE_BIT31 31
#define TYPE_BIT(n) (n)
#define TYPE_BYTE 32
#define TYPE_INT 45
#define TYPE_FLOAT 78
static inline uint32_t config_hash(const char *src)
{
uint32_t result = 5381;
while (*src)
result = (result * 33) ^ (uint32_t)(*(src++));
return result;
}
/* Example items */
float origin_x;
float origin_y;
float origin_z;
int speed_v;
struct {
int a;
int b;
int c;
} frob;
/* Configuration array, naming all configurable variables. */
static const struct config_entry config[] = {
/* Keep sorted by vvvvvvv */
{ "origin.z", &origin_z, 5255461, TYPE_FLOAT },
{ "origin.y", &origin_y, 5255462, TYPE_FLOAT },
{ "origin.x", &origin_x, 5255463, TYPE_FLOAT },
{ "speed.v", &speed_v, 6516090, TYPE_INT },
{ "frob.b", &(frob.b), 15471920, TYPE_INT },
{ "frob.c", &(frob.c), 15471921, TYPE_INT },
{ "frob.a", &(frob.a), 15471923, TYPE_INT },
};
The config[] array is in ROM/Flash. When you add new entries, you do need to calculate the 24-bit hash for the third column; but you can use for example the following hash.py Python script:
from sys import argv, stdout, stderr, exit
def hash(string):
result = 5381
for c in string:
result = ((result * 33) ^ ord(c)) & 4294967295
return result & 16777215
if __name__ == '__main__':
if len(argv) < 2 or argv[1] in ('-h', '--help', '/?'):
stderr.write("\n")
stderr.write("Usage: %s [ -h | --help | /? ]\n" % argv[0])
stderr.write(" %s STRING [ STRING ... ]\n" % argv[0])
stderr.write("\n")
stderr.write("This program computes the 24-bit DJB2 XOR hash\n")
stderr.write("of the input strings, and outputs them as\n")
stderr.write("a C hash table array.\n")
stderr.write("\n")
exit(0)
table = []
for arg in argv[1:]:
table.append((hash(arg), arg),)
table.sort()
for entry in table:
stdout.write(" { \"%s\", NULL, %8d, 0 },\n" % (entry[1], entry[0]))
At run time, you can use a binary search to rapidly locate the correct configuration item:
const struct config_entry *find_config(const char *name)
{
const uint32_t hash = config_hash(name);
const int_fast32_t n = (sizeof config / sizeof config[0]);
int_fast32_t imin = 0, iend = n;
int_fast32_t i;
if (hash < config[0].hash || hash > config[n-1].hash)
return NULL;
while (iend > imin) {
i = imin + (iend - imin)/2;
if (config[i].hash < hash)
imin = i;
else
if (config[i].hash > hash)
iend = i;
else {
/* Extend imin, imax to cover all matching hashes */
imin = i;
while (imin > 0 && config[imin-1].hash == hash)
imin--;
iend = i + 1;
while (iend < n && config[iend].hash == hash)
iend++;
break;
}
for (i = imin; i < iend; i++)
if (config[i].hash == hash && !strcmp(name, config[i].name))
return config + i;
return NULL;
}
From there, it is just a matter of examining the ->type, and getting or setting the corresponding value from ->addr.
On a 32-bit microcontroller, this takes 13 bytes plus the name string per configurable variable of ROM/flash.
Note that you can integrate the hash calculation to your parser, so that whenever you find an identifier, you have both the identifier string and its hash.