Hello,Why do you need 24 byte long filenames? Can't you store the name somewhere else and use shorter names?
I would like to cache 512 8KB files on an SD card. These files are only of interest to the microcontroller and are write-once/read-often. I am currently placing them on a regular FAT32 partition but the single filenames are 24 bytes and would require "Long filename" or to be divided in folders. Sadly, tonight's test of creating many folders and looking up the files deep in the hierarchy is slow.
So I would like to use another file system in another partition, eventually code it. I was previously using an SPI flash module with a simple lookup table. I was thinking of shrinking the card FAT32 partition and recreate the SPI flash principle in the raw zone. The FAT32 partition should still be accessible by the microcontroller and exposed through USB-MSC. Is there a risk to this ? First thing which comes to mind is that users could re-format the USB-MSC drive as one single partition and override the second "raw zone".Can't you lock the raw sectors in the firmware by simply telling a smaller size to USB to avoid writing into the raw zone? Or creating multiple partitions and only forwarding the main partition to USB?
What sort of data are you storing?The regular FAT32 partition stores a 4GB file of which only few blocks need to be read by the microcontroller. Once processed, the result is stored in these 8KB files and later retrieved as pre-computed.
I'm not sure what your issue is with long filenames, fat32 supports them.The issue is licensing it from Microsoft or having to implement it myself. In which case I'd rather implement the solution below.
Why do you need 24 byte long filenames? Can't you store the name somewhere else and use shorter names?The unique identifier of each file is 24 bytes.
If you write the date once, you could emulate the SPI flash style using a single file on the drive with your table and all 512 8kB sections.
Can't you just concatenate all 512x8kB files into a single 4MB file on the partition root?I will try this today.
I think you missed the point. You don't need 24 bytes (192 bits) of filename to distinguish just 512 files (it takes only 9 bits). In other words, just use one Table of Contents file (e.g. cache.toc) containing an array that associates each 24 byte UID to a file with a short name (e.g. cache001.dat). This completely avoids the need to store the UID as the file name. Instead of dicking around with Microsoft long file name licensing, just use this lookup array. Easy.Why do you need 24 byte long filenames? Can't you store the name somewhere else and use shorter names?The unique identifier of each file is 24 bytes.
open cache file
store cache index from first 8192 bytes to memory
for(i from random+0 to random+16)
seek to location from cache index [i]
read 8192 bytes of interest
close cache fileopen cache index file
store cache index to memory
close cache index file
for(i from random+0 to random+16)
open cache file at path from cache index [i]
read 8192 bytes of interest
close cache fileThe issue is licensing it from Microsoft ...
It is impossible to give you an answer without knowing more about the purpose of your cache system.
I would use a single file:
A typical cache should be fast, therefore the index should be kept in ram. I assume you need an index (the 24 byte long name). Maybe you can calculate the index directly from the 24 byte name without any additional table.
From the index you get the address in memory (= in the cache file). If you keep the file open all the time you can seek very fast to a specific location and read the desired entry.
If you allocate the cache file at once on an empty card it will not be fragmented, so the filesystem does not need to jump in the cluster chain.
You can also increase the cache file anytime if needed, but it will fragment the file if other files have been created and make it slower.
Opening different files is slower in most cases because the file system drives needs to go through the directory list and look for the specific filename. But it highly depends on the filesystem driver and the amount of data it buffers in ram.