best approach to compare/evaluate data received on UART

[sairfan1]

Hi, I'm looking for a good approach so that i can develop some method which can help me to compare/evaluation what command/data i received on UART.  While learning on internet i came accross State Machine approach, I was successfull to evaluate received data of length 3/4 chars.  But switch statement becomes so messy and complex if we keep adding more conditions.

For example when i came accorss the situations like working with GSM modules where we have a lot of AT commands and response, its not possible to compare all of them in switch statement.  In some other scenarios we may receive 20/30 char long array again its very hard to evaluate what did we receive.

I was looking for approaches used in such scnarios so that i can learn.

Thanks.

[stmdude]

This is purely a software question, so maybe this isn't the best forum, but I'll give it a shot.

Instead of a massive switch-statement, you might consider an array of response prefixes and only match when you've received a whole line. (This is valid for processing AT commands, but maybe not other protocols).

Something along the lines of (I'm assuming you're using C, and compiler capable of at least C99):

Code: [Select]

typedef struct {
 char *prefix;
 int (*cb)(char *line);
} prefix_cb_t;

int response_OK(char *line) { printf("I got a OK\n"); }
int response_ERROR(char *line) { printf("I got a ERROR\n"); }
int response_CPIN(char *line) { printf("I got a +CPIN\n"); }

prefix_cb_t myprefixes[] = {
 { .prefix = "OK", .cb = response_OK },
 { .prefix = "ERROR", .cb = response_ERROR },
 { .prefix = "+CPIN:", .cb = response_CPIN },
 { .prefix = NULL, .cb = NULL },
};

int process_line(char *line) {
 int i;
 while(myprefixes[i].prefix != NULL) {
  if( strncmp(line,myprefixes[i].prefix,strlen(myprefixes[i].prefix)) == 0 ) {
   return myprefixes[i].cb(line);
  }
  i++;
 }
 printf("\"%s\" didn't match any known prefixes\n",line);
 return -1;
}

void main(void) {
 process_line("OK\r\n");
 process_line("ERROR\r\n");
 process_line("+CPIN: OK\r\n");
}

[sairfan1]

Do you think its efficent way when we have about 100 possible responses, and we expect one after an other?
Thanks.

[TomS_]

I did something similar in a project last year.

I allocated 16 bytes of memory to an array of chars, my "receive buffer", as I wasnt expecting any legitimate input to exceed this length.

Each byte that was received by the UART was appended to the array, and upon receiving the appropriate termination character I compare the received string to a series of expected strings using the strncmp function (I was also using C). Once receiving the 16th byte, if that would ever happen, an error would result because then I would be overwriting the allocated space (basically I just wrap around and overwrite the earlier received data - it probably wouldnt match anything I was expecting, so it would end up discarded anyway).

You could also implement your own very simple string comparison routine if you wanted to, much like what stmdude has done.

You can still implement a state machine and have it reasonably efficient I suppose. When the state machine is "reset", a variable storing the state is at zero. When you receive a string and the state machine does its first pass it will switch to case 0 and compare the expected strings that you would want to find at state 0. If you find one you increment the state variable, or set it to the appropriate value so that on the next iteration of the state machine it switches to the appropriate case which then compares the next set of expected strings.

Thats the best I can come up with anyway.

[sairfan1]

Is there some other method particularly used to parse what we receive on serial port, just like state machine, as my major concern is AT commands and there response, is there some particular algorithm/technique used to parse such data?

[cdev]

I have used SQL databases to handle these kinds of things. Its not as overkill as it sounds because they have DBs that run in only a few MB of memory and are quite efficient in using resources. You don't have to save anything to disk with some DBs. They can run in memory.

Its fairly simple, you need to develop a schema that shows each possible output and the possible variations on it, You save each variation to its own table. PostgreSQL with its object-relational tables lends itself well to this.

Look for software that parses things like GPS NEMA sentences for examples of code. You dont have to use a real database, just using a database like approach.

Just my two cents.

[rstofer]

It would seem to me that the possible responses to any particular AT sequence are pretty limited.  Further, there may be a limited set of AT commands used in any given application.

I would write a function (subroutine) for each command I used and have code there to handle the responses.  I could #ifdef them in and out of the executable depending on what I need.

You might need to include pointers to structs that the function will fill out from the response.

In the end, it's going to be a lot of code.

[Ian.M]

The most efficient for a large set of moderate to long commands is to prebuild a static B-tree which at each level has a child node for the next possible character in the command keyword.  The root node has children for each possible first character.   The leaf nodes have pointers to the different command action routines.  Then as each character is recieved walk the tree.  Each action routine calls subroutines to fetch its expected parameters then performs the command.

However its a royal PITA to prepare the B-tree structure manually, so you generally need tp write a program on your host system to accept a list of commands and their action routine names and generate the sourcecode required,

A much simpler system that searches a list of command names with a separator character between them for a match  then uses the match position as a command code in a switch statement may be preferable for small command sets or if speed isn't a concern.  You still need to generate enums for each command position to use in the switch, but that's simple enough to do in a spreadsheet  There's also the array of keywords as suggested by stmdude above, but that wastes a lot of space if you have mostly short keywords and a few long ones.  If you are short on space, you'll have to do some test builds to see which is the best approach.

[Naguissa]

Hello,


I made a library for working with ESP8266 AT commands, time ago.

I suppose it's quite similar, the only problem is that library is more oriented as web server.

You can take a look here: https://github.com/Naguissa/WiFiLib


Also, created an even more http server oriented one, dealing with SD card and few clients at same time using a cooperative mode implemented with state machines:

https://github.com/Naguissa/WiFiSDCoopLib


Still, as they are AT-commands based, I think you can get some ideas from them.


Cheers.

[sairfan1]

I was looking for some solution that I can use in different project.

@Lan.M's method looks interesting, I understand searching each received char will help us to validate what char we received, but at end how will we evaluate what command it was? do we nead to calculate some value that finally end up to some ID that represents some specific command/response?

For example we received one by one chars RST and validated each char R -> S -> T at this point we have been validated all chars (they found in our list) but it does not indicate some identity that what it was, (in reall scenario it will be a longer char array and we are not compairing strings).

@Naguissa thanks for the code sharing, I'm currently doing almost same, problem with this approach (what i understand) is that it can't be used in different projects for example you will have to write a separate logic for GSM, and also if there are too many responses that you have to manage you end up with a mess.  It sounds like for b-tree approach we will have to build a table of possible response, rest of things will be done by library written for this task.

[TomS_]

Receive your bytes in to a buffer, wait for a termination sequence to be received (e.g. CRLF), and then process the string in its entirety afterwards. Its basically how everything works because you then have the full state and contents of the request or response to examine at will.

This can still be tied in to a state machine to ensure that the correct sequence of commands/responses are transmitted/received. You just wont be processing in real time, byte by byte and trying to keep track of where you are in respect to a particular sequence of bytes which should make things much more simple.

You might only look at the first 4 bytes, for example, to see if that matches any of the 4 byte commands/responses you are expecting, or maybe you look at 5, or more, or some combination. It all depends on what you are looking for to verify a correct command/response at any given state of your state machine. If there are "parameters" contained within the message as well, you should be able to find them by parsing the string and picking out the pieces which live between the appropriate delimiter character.

Its going to be fiddly and require effort, but thats what you get with microcontrollers.

If what you want is something more cookie cutter and ready to go, you might want to look at an Arduino or maybe a Raspberry Pi or similar where there may be other libraries or languages available that are easier to work with and require you to do less ground work to get your project up and running.

[Kalvin]

Check the first character of a response and jump to a function which will then determine the actual command. Using a table of function pointer will be faster than switch-case. Depending on your speed requirements you can implement another level of jump tables, switch-case or if-then-else chains. Alternatively you can hash the command using a very simple hash function and use the computed hash as an entry to a lookup table containing function pointer to next level compare functions. In this way you can reduce the time spent in long if-then-else chains.