Microcontroller block cache for SD card access

[bernroth]

Hello all!
I have an embedded system with an 16bit MCU which uses a SD card to store and read data.
I found out that the access to the SD card is performing much better when implementing a cache layer in-between.
Well, my first implementation was working but sometimes I get file system corruption.

I have been looking through the sources of the Linux kernel but these are using just too much dynamic memory allocations etc.
As you know on a MCU the code should be as efficient as possible and use static memory allocation (caching e.g. 8x 512byte blocks should be just fine).

Debugging and error finding is a bit tricky on that target system so I wonder if anyone can point me to a (working) implementation of a simple MCU compatible read/write block cache.

Any hints are welcome - thank you!

Best regards,
Bernhard

[rsjsouza]

Depending on the device, you may be able to use background transfers via DMA (check this app note), which would help with the overall performance of the system.

I also found this discussion about FatFS as well, which may help with your own project.

[marshallh]

For my own application I cached 2 sectors worth, and simply used them in ping-pong LRU replacement. This alone greatly improved the speed of traversing the FAT.

With 8 sectors and even a basic round robin replacement you should get vastly improved speeds.


1. Check if cache 0-7 contain the sector already
2. If so, just use it
3. If not, load the block into the current cache pointer, then increment the index so next time the next slot is replaced

When you have more RAM you can think about doing stuff like FAT caching, read-ahead, or even just whole-block caching (which most SD cards already do - NAND block size is many times larger than 512 bytes)

[bernroth]

I have been going though various sources and your hints and came eventually to a working prototype.

What I adopted is a kind of LRU cache mechanism which uses a fixed buffer of n blocks which is allocated at startup of the SD card driver.
Fixed buffers allow to avoid memory fragmentation and slowness due to malloc/free operations.

When a cache block is accessed or modified, an internal timestamp keeps track of the age of each entry. I do not use some internal real time clock but simply a counter which increments by 1 each time I get a new timestamp. Eviction of old cache items is done automatically by choosing the oldest one for new items. I avoided the use of linked lists as I keep only around 32 blocks in memory.
WB cache is implemented as well and it can occupy at most n % of the total available cache memory (n blocks). Write cache is flushed to disk by a separate task either after 200ms or when the maximum threshold is reached.

The additional layer of code is noticeable when doing I/O performance tests on single files as results are worse than without caching.

Anyhow the big advantage is that now access to the FAT or RMW operations (done quite often by the OS when calling fwrite with few data) are much faster and the overall system performance is a lot better.

Thanks for your help and incitations 

Now I have to test, tune and improve my code