EEVblog Electronics Community Forum
Electronics => Microcontrollers => Topic started by: ali_asadzadeh on March 28, 2019, 10:24:21 am
-
Hi,
I have captured some data from ADC with my FPGA using UART, the output captured data is like this, I want to know if there is a readily available tool to convert the data which is in hex into int32 so that I can plot the data using matlab.
Note that the data is 24bit and I have added FF to separate the data, I can change the data format to 32bit by replacing the FF with 00, also the data is a signed data.
-
That is not an HEX (intelHEX) file.
Just refactor the text using simple C program, written for purpose. Would get you somewhere within couple of tens of minutes. Parsing this is very easy. (even using directly matlab I'd say).
//EDIT: Also please note, that by "just adding FF or 00" in front of a 24bit two's complement (signed) number does not cut the mustard. You must properly sign extend.
FF FC B5 EE is correct, but FF 01 C4 6C is not, should be 00 01 C4 6C. Copy the uppermost bit into the preceding byte.
-
//EDIT: Also please note, that by "just adding FF or 00" in front of a 24bit two's complement (signed) number does not cut the mustard. You must properly sign extend.
FF FC B5 EE is correct, but FF 01 C4 6C is not, should be 00 01 C4 6C. Copy the uppermost bit into the preceding byte.
Thanks for the hint, I'm aware of it ;)
Just refactor the text using simple C program, written for purpose. Would get you somewhere within couple of tens of minutes. Parsing this is very easy. (even using directly matlab I'd say).
I didn't want to write a c program for it, because I think it should be probably something out there that would do the job.
-
Yes, there is, for example the matlab to begin with ;)
https://se.mathworks.com/matlabcentral/answers/312599-how-do-i-parse-this-complex-text-file-with-textscan
It is as easy to do as in C. /* The C being likely the easier way*/
-
If you have a unix shell available (and chances are very high, among W10, Linux and macOS) you can use something as:
xxd -r -p dump.txt | od -t u4 -An
on the raw hex data (no header or footer: tail can come to the rescue).
This assumes the 24 bit data is big endian, and does not correct the sign and magnitude (I think that is easier in matlab)
man xxd (https://linux.die.net/man/1/xxd)
man od (https://linux.die.net/man/1/od)
Still, I would write a short C program...
-
I'm surprised so many suggest C. Isn't this exactly what your average scripting language is for? Python, PHP, Pike, JavaScript, Perl etc etc.
Anyhow, a quick custom piece of code is probably the way to go so that you have easy control over the parsing and the output.
-
How are you going to use the converted data? Read it into some sort of program? Then just modify your program to include parsing. It's 10 minutes of work.
-
I'm surprised so many suggest C. Isn't this exactly what your average scripting language is for? Python, PHP, Pike, JavaScript, Perl etc etc.
You are probably right: I was thinking of doing it in PowerShell just for kicks...but then in a 10 minutes break I concocted this (use as a filter on command line):
#include <stdio.h>
#include <string.h>
#define SIZE 1024 /* Maximum allowed line length */
char lineBuffer[SIZE];
int main(void)
{
/* Skip header - 3 lines */
for (int i = 0; i < 3; i++)
fgets(lineBuffer, SIZE, stdin);
/* Stop at first short line (or EOF) */
while(fgets(lineBuffer, SIZE, stdin) && (strlen(lineBuffer) > 2))
{
char *s = lineBuffer;
unsigned short int l, m, h;
/* Parse groups, ignore delimiter value, stop at first problem */
while(3 == sscanf(s, "%*X %hX %hX %hX", &h, &m, &l))
{
/* Assume big endian */
int value = (h << 16) + (m << 8) + l;
if(value >= 0x800000)
value -= 0x1000000; /* Adjust negative numbers */
printf("%d\n", value); /* Write output */
s += 12; /* Next group */
}
}
return 0;
}
It's throwaway code, not much error checking or style, HTH though...
-
As mentioned above, there is a simple MATLAB program for doing this. But MATLAB costs money so maybe it can be ported to Octave.
Octave has hex2dec() so you can do something like hex2dec(ff) and get back 255. I didn't pursue it further...
-
I'm surprised so many suggest C. Isn't this exactly what your average scripting language is for? Python, PHP, Pike, JavaScript, Perl etc etc.
Anyhow, a quick custom piece of code is probably the way to go so that you have easy control over the parsing and the output.
If you're working with embedded stuff, chances are that you're familiar enough with C to do it easily.
Bit/byte manipulation is notably not very straightforward in scripting languages, because they were designed for higher-level tasks.
Here's my 3-minute attempt:
#include <stdio.h>
main() {
int a, b, c, d;
while(scanf("%x %x %x %x ", &a, &b, &c, &d) == 4) {
a = b & 128 ? 255 : 0;
a = (a << 24) | (b << 16) | (c << 8) | d;
fwrite(&a, 1, 4, stdout);
}
}
It would take me much longer to search for an existing program to do it.
-
Bit/byte manipulation is notably not very straightforward in scripting languages, because they were designed for higher-level tasks.
You have obviously been using the wrong scripting languages then. :) Sane ones are pretty much the same as C for those operations.
-
Here's how you would do it in PHP :
In Windows, just download php in a folder, for example C:\Programs\PHP and then open a command prompt in the same folder with your php script and text file and say
c:\programs\php\php.exe script.php
Change file name in the script if you want. Script just dumps each number on one line. You can say php.exe script.php >output.txt to redirect console output to a text file.
You can modify the script so that instead of a new line, the script prints a "," or tab ("\t") or whatever character you want.
<?php
$ENTER = "\x0D\x0A"; // Windows uses CRLF , Linux and only one character... change this as needed
// determine the path from where the script runs , and change all \ slashes to / slashes - Windows can work with either.
$folderRoot = str_replace('\\','/',dirname(__FILE__));
if (substr($folderRoot,strlen($folderRoot)-1,1)!=='/') $folderRoot .='/';
// change the file name here or replace with $argv[1] to read the file name from command line ex php.exe script.php filename.txt -> $argv[1]=filename.txt
// though it will only work if files have no spaces in their names
$content = file_get_contents($folderRoot.'20190328142916.txt');
$lines = explode($ENTER,$content);
$v = array();
$vindex = 0;
foreach ($lines as $line) {
if (isHexLine($line)==TRUE) {
$values = explode(' ',$line);
foreach ($values as $value) {
if ((ctype_xdigit($value)==TRUE) && (strlen($value)==2)) { // hexadecimal , one byte
$nr = hexdec($value);
$v[$vindex] = $nr;
$vindex++;
if ($vindex==4) { // read 4 bytes
// comment next line if you don't want the first byte to be reset
$v[0] = 0;
$nr = ($v[0]<<24) + ($v[1]<<16) +($v[2]<<8) + $v[3];
echo $nr.$ENTER;
$v[0] = 0; $v[1]=0;$v[2]=0;$v[3]=0;
$vindex=0;
}
}
}
}
}
// function checks if the line of text is composed of only hexadecimal characters or space
function isHexLine($text) {
if (strlen($text)==0) return FALSE;
for ($i=0;$i<strlen($text);$i++) {
$c = substr($text,$i,1);
if (($c==' ') || (ctype_xdigit($c)==TRUE)) {
// it's space or hexadecimal char, so continue
} else {
return FALSE; // bad char, don't process this line
}
}
return TRUE;
}
?>
-
Wow...really thanks guys, I did not expect that, really appreciate the tips. though I'm a fan of Node and Javascript
-
So for completeness I have written the Node version too
<?php
const fs = require('fs');
const EOL = require('os').EOL;
const contents = fs.readFileSync('./20190328142916.txt');
const reg = /FF\s+(\w{2})\s+(\w{2})\s+(\w{2})/g;
let group = reg.exec(contents);
let combinedDecimalValues = '';
while (group) {
const hexValue = `${group[1]}${group[2]}${group[3]}`;
const parsedValue = parseInt(hexValue, 16);
let decimalValue = parsedValue;
if (parsedValue > 0x7FFFFF) {
decimalValue = parsedValue - 0xFFFFFF - 1;
}
combinedDecimalValues += decimalValue + EOL;
group = reg.exec(contents);
}
fs.writeFileSync('./output.txt', combinedDecimalValues);
mariush how did you highlighted your code?
Do we have code highlight in here?
-
That's how:
<?php
const fs = require('fs');
const EOL = require('os').EOL;
const contents = fs.readFileSync('./20190328142916.txt');
const reg = /FF\s+(\w{2})\s+(\w{2})\s+(\w{2})/g;
let group = reg.exec(contents);
let combinedDecimalValues = '';
while (group) {
const hexValue = `${group[1]}${group[2]}${group[3]}`;
const parsedValue = parseInt(hexValue, 16);
let decimalValue = parsedValue;
if (parsedValue > 0x7FFFFF) {
decimalValue = parsedValue - 0xFFFFFF - 1;
}
combinedDecimalValues += decimalValue + EOL;
group = reg.exec(contents);
}
fs.writeFileSync('./output.txt', combinedDecimalValues);
:-DD
In general, you can click Quote to obtain "source code" of other people's posts.
-
I'm surprised so many suggest C. Isn't this exactly what your average scripting language is for? Python, PHP, Pike, JavaScript, Perl etc etc.
Nothing surprising here. It's a 5-minute throwaway job on any half-decent language. It's approx 10-20 lines on any language, Python or C.
The result (how easy and fast it is) depends entirely on the language skills and the current mindset of the programmer. If they have written C for an embedded project in last years, trying to do it in Python because it's somehow "easier" could end up being a disaster in time usage.
Really almost any language will do. Trying to find some academical "optimally easy" solution doesn't make a lot of sense if it isn't easy in your specific case.
When you have a round peg and a round hole and need to apply just a little bit of extra force to get it in, it doesn't matter much which kind of hammer you end up using, most will do, and you'll use what you have already in your hand, as long as it isn't clearly wrong. Shopping for an optimum type of hammer would make zero sense.
For a beginner, Python - as well as many other scripting languages - are good choices, if they don't know any language yet, and need to start by learning the language and the tools.