Electronics > Projects, Designs, and Technical Stuff

On the fly data compression with 32 bit uC

(1/4) > >>

sgpee:
Hi,

Do you guys have any experience with on the fly data compression? I am not talking about complex huffman encoding, something simpler but still gives me 20-30% savings and it can operate on fixed boundaries? (i.e. 256 bytes in x bytes out, but always 256 in)

FYI. My data is fairly linear, so the 256 byte will have a nice uniform pattern, should be pretty easy to compress with the right algo.




Thx,
sgpee

Mechatrommer:
use fpga.

Zad:
The simplest would be run-length encoding (if your data is suitable) http://en.wikipedia.org/wiki/Run_length_encoding

Lempel Ziv Welch (LZW) is pretty efficient in terms of CPU time too http://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch

oPossum:
You may be able to encode the change in data rather than the absolute value.

I did this with a data logger that read a 12 bit ADC and stored the samples as 8 bit deltas (-127 to +127). Each block of 1024 bytes had the absolute value of the first 12 bit sample, 999 8 bit deltas, and a timestamp.


--- Code: --- delta = adc - prev_adc;
if(delta < -127) {
delta = -127;
} else if(delta > 127) {
delta = 127;
}
prev_adc += delta;
*buffer_ptr++ = (uint8_t) delta;

--- End code ---

complete source code

allanw:
Have you thought about using SPI to write to an SD card? Then you can have virtually unlimited storage.

Navigation

[0] Message Index

[#] Next page

There was an error while thanking
Thanking...
Go to full version