Electronics > Projects, Designs, and Technical Stuff

On the fly data compression with 32 bit uC

<< < (4/4)

sgpee:
It is easy to make this algo quasi lossless. (one value will be gone)

My data is 16 bit and if I am willing to let go of one value say 0xff than that would act as a marker. In this case, if you are outside the boundary you insert marker and reset the start value to the extreme and go from there. I need to check the values to understand how well this would work but my hunch tells me, this would easily save 20-30%.


sgpee


--- Quote from: oPossum on November 24, 2010, 07:18:59 pm ---
--- Quote from: TheDirty on November 24, 2010, 04:50:20 pm ---It would not be hard to make it lossless with an RLE type scheme, if you really needed it.

packet length | start value | delta | delta | delta...

--- End quote ---

Hmm. Inspiration for this concept...

marker | absolute | delta | delta | delta...

The marker would be a special value outside the delta range. For example a signed 8 bit value can be -128 to 127. So use -127 to 127 for delta, and -128 for marker.

No limit on run length with this method. And no streaming problems (no need to know packet length at head of data block).


--- End quote ---

sgpee:
Mike,

I am not familiar with barrel-shifter, would you care to elaborate?

Thx,
sgpee



--- Quote from: mikeselectricstuff on November 24, 2010, 10:03:41 am ---For  data which is amenable to difference coding but may contain occasional spikes, variable-length coding can work well, e.g. reserve some code values or bits to indicate special-cases of absolute values or higher deltas.
If using an ARM, the barrel-shifter can make variable-length coding very efficient.
 

--- End quote ---

Mechatrommer:

--- Quote from: sgpee on November 25, 2010, 09:17:16 am ---It is easy to make this algo quasi lossless. (one value will be gone)

--- End quote ---
it is easy as well to do it purely lossless as previously discussed. just simple test of if greater than 127 or less than -127 will throw out -128 (marker) and absolute adc value. the problem will be just it will expanded instead of compressed for highly oscillating adc.

another way is using table based compression. i have here a fin table prediction compression method for pc file, not mcu unfortunately (not sure where i got the "fin" name, its been years ago). it is alot simpler compared to lzw. but it will more codes to do the setup and housekeeping and only suitable for highly stable and repeatative data. it is possible to compress up to 1bit per data for a highly predictable data.

mikeselectricstuff:

--- Quote from: sgpee on November 25, 2010, 09:18:30 am ---Mike,

I am not familiar with barrel-shifter, would you care to elaborate?

Thx,
sgpee



--- Quote from: mikeselectricstuff on November 24, 2010, 10:03:41 am ---For  data which is amenable to difference coding but may contain occasional spikes, variable-length coding can work well, e.g. reserve some code values or bits to indicate special-cases of absolute values or higher deltas.
If using an ARM, the barrel-shifter can make variable-length coding very efficient.
 

--- End quote ---

--- End quote ---

Most ARM instructions can shift or rotate one of the operands by any number of bits with no extra execution time, so doing things like packing variable-length bitfields into 32-bit words can be done very effciently. It can also shift data by a number of bits specified in a register.
e.g. in C, the following lines can each be done as a single ARM instruction:
a=b<<5|c;
b<<=c;

Assuming of course the compiler is suitably aware of the archtecture.

Mechatrommer:

--- Quote from: mikeselectricstuff on November 25, 2010, 10:51:03 pm ---Most ARM instructions can shift or rotate one of the operands by any number of bits with no extra execution time, so doing things like packing variable-length bitfields into 32-bit words can be
--- End quote ---
pic and avr also have this bit shift feature iirc, only a bit at a time/op... asm'wise.

Navigation

[0] Message Index

[*] Previous page

There was an error while thanking
Thanking...
Go to full version