Electronics > Microcontrollers

More compression codecs

<< < (3/3)

SiliconWizard:
I've played with QOI some more.

While it performs rather well, and not too far from PNG for a whole range of images (including the author's test set  ;D ), it performs pretty poorly for some of them. In particular, I have images generated with complex textures, and the compression ratio falls to just about 1/2, while the PNG equivalent is almost 1/10 of that!

But! That wasn't the end of it. I looked for a simple and fast compression that I could apply to QOI (a bit in the same vein as PNG does some pre-filtering, and then uses - I think - 'deflate' to compress it). I found XLZ, which is a simple and fast LZ77 compression: https://github.com/banebyte5115/xlz . It does happen to complement QOI pretty well, and while both QOI+XLZ is still MUCH faster for encoding (and even decoding) than PNG, it does perform very well on a wider range of images than just QOI, and the cumulated code is just a few KBytes. The above images I mentioned, which were "tough" for QOI alone, get compressed BETTER than the highest-level PNG.

While QOI can easily be turned into a streaming compression and decompression, XLZ is another beast, though. As it's LZ77, it does require a significant amount of memory - basically, it requires a "sliding window". But you can always work around that for memory-limited applications by compressing data in smaller chunks. Might not be quite as efficient compression-wise, but it's workable.

So, I'm definitely considering chaining the two for some applications.
I suggest considering the source code in both projects as "reference implementations" (as the author of QOI states), rather than production-ready code, and thus suggest rewriting those with your own constraints, code style and coding rules, if they apply, for any serious project.

Smokey:
LZ4 looks to use a default block size of 64k, but I've seen claims it works down to 1024 with lower compression ratios.  The LZ4 python bindings appear to ignore you when you try to set a block size less than 64k, so I gave up on that for the moment and I'm working with heatshrink since small blocks is upfront in the feature set. 

SiliconWizard:
I've tried LZ4 before XLZ, but XLZ on QOI (I'm not talking about any data in general) works almost as well as LZ4, and is much simpler (and even faster).
Both will have the same kind of memory footprint and constraints, so using that depends on your target and requirements entirely. Probably not worth it for small targets, but otherwise interesting.

Haven't tested heatshrink. Could you give examples of compression ratios with it on example images and compare that to QOI alone?

T3sl4co1l:
Necroposting:

A new, old algorithm for a change, has been published here: https://github.com/TheRealOrange/icer_compression

Also cross referencing these pages for convenience:
https://www.eevblog.com/forum/programming/slic-simple-lossless-imaging-codec/msg4723580/#msg4723580
https://www.eevblog.com/forum/microcontrollers/best-libraryalgorithm-for-low-ram-embedded-data-compression/
https://www.eevblog.com/forum/microcontrollers/arithmetic-decoding-for-embedded/
and a few other threads we've been through here that I recall but don't find at a glance.

Tim

Smokey:
Cool.  They mention "memory-constrained embedded systems".  But they don't give hard numbers on that, and their examples uses a ton of memory.

From their example: https://github.com/TheRealOrange/icer_compression/blob/master/example/src/main.c


--- Code: ---   
    const size_t out_w = 512;
    const size_t out_h = 512;

    uint8_t *resized = malloc(out_w*out_h);
    uint16_t *transform = malloc(out_w*out_h*2);
    uint16_t *compress = malloc(out_w*out_h*2);
    uint16_t *decompress = malloc(out_w*out_h*2);
    uint8_t *display = malloc(out_w*out_h);

--- End code ---

So decompress needs 512*512*2 uint16_t?  That's 1,048,576 bytes? 

Navigation

[0] Message Index

[*] Previous page