Advice for filesystem in Nand Flash

[iamno3]

I'm currently building emergency panel that contain a 128 MB spi nand flash. Which is store 4 emergency voice WAV/MP3 format. User can change default emergency voice. Just connect the emergency panel via usb to windows PC.
I'm using FAT filesystem on nand flash. There are still some bug. Copy file is really slow since nand characteristic need to copy whole block to replace one page but Windows keep changing boot sector and FAT table many times for copy just 1 file. So it take much longer for several hundred KB.
Is there any other optional file system that work with windows. Or is there any commercial file system ?
I use no OS.

[ataradov]

There is no other file system that would be better in that respect. The solution here is caching - just save the often modified pages in SRAM and flush them at the end of the write.

Another common solution is to use something like UF2 file format and basically ignore all the writes that don't contain a valid signature. And the ones that do contain the signature would trigger a direct write. This way only relevant information would be transferred and the drive would be virtual. But that would require a separate tool to convert your files into this new format. On the plus side, you would not need a FatFs on the MCU side to read the files back, since they would be stored well known locations.

But doing the caching is probably the most user friendly way.

[fchk]

Another way is not to use USB Mass Storage and a filesystem but MTP like many MP3 players or phones.

https://en.wikipedia.org/wiki/Media_Transfer_Protocol

MTP only transfers raw file data plus metadata - the PC does not see the underlying storage layer. This allows you to use an optimized storage method with or without filesystem.

fchk

[cgroen]

I'm currently building emergency panel that contain a 128 MB spi nand flash. Which is store 4 emergency voice WAV/MP3 format. User can change default emergency voice. Just connect the emergency panel via usb to windows PC.
I'm using FAT filesystem on nand flash. There are still some bug. Copy file is really slow since nand characteristic need to copy whole block to replace one page but Windows keep changing boot sector and FAT table many times for copy just 1 file. So it take much longer for several hundred KB.
Is there any other optional file system that work with windows. Or is there any commercial file system ?
I use no OS.

Segger has MTP support, I would avoid FAT/MSC if at all possible!

[iamno3]

There is no other file system that would be better in that respect. The solution here is caching - just save the often modified pages in SRAM and flush them at the end of the write.

Another common solution is to use something like UF2 file format and basically ignore all the writes that don't contain a valid signature. And the ones that do contain the signature would trigger a direct write. This way only relevant information would be transferred and the drive would be virtual. But that would require a separate tool to convert your files into this new format. On the plus side, you would not need a FatFs on the MCU side to read the files back, since they would be stored well known locations.

But doing the caching is probably the most user friendly way.

I try caching solution it was sound easy but I just realize my MCU RAM are 64 KB, erase per block NAND are 128 KB. Need to change MCU but it will take time again to redesign the pcb and order the part.

Another way is not to use USB Mass Storage and a filesystem but MTP like many MP3 players or phones.

https://en.wikipedia.org/wiki/Media_Transfer_Protocol

MTP only transfers raw file data plus metadata - the PC does not see the underlying storage layer. This allows you to use an optimized storage method with or without filesystem.

fchk

Segger has MTP support, I would avoid FAT/MSC if at all possible!

Shoot !! this is good solution but there is no usb MTP class implementation for renesas MCU 
I'm thinking to buy FTL code/library from this site to keep up with project schedule

Code: [Select]

https://zeeis.com/flash-translation-layer/

is this good ?

also some other solution
> change to 1Gbit NOR Flash but it will cost more
> change MCU with larger RAM but it will take more time

[ataradov]

I try caching solution it was sound easy but I just realize my MCU RAM are 64 KB, erase per block NAND are 128 KB. Need to change MCU but it will take time again to redesign the pcb and order the part.

Don't cache flash sectors, cache USB MSD 512 byte blocks.  It is hard to tell for sure what caching strategy would work, but I can't imagine that a simple file system with a couple files would have a lot of blocks with meta information.

Also, how do you do updates now? You will get information in 512 byte chunks form USB and eventually you would have to erase and update the whole 128 KB block.

[iamno3]

I try caching solution it was sound easy but I just realize my MCU RAM are 64 KB, erase per block NAND are 128 KB. Need to change MCU but it will take time again to redesign the pcb and order the part.

Don't cache flash sectors, cache USB MSD 512 byte blocks.  It is hard to tell for sure what caching strategy would work, but I can't imagine that a simple file system with a couple files would have a lot of blocks with meta information.

Also, how do you do updates now? You will get information in 512 byte chunks form USB and eventually you would have to erase and update the whole 128 KB block.

Hi Alex, I choose to use NOR solution. I think it's more simple to implement on software. No need thinking about bad block management. And erase block is small only 4 KB.
Thinking about strategy for caching 512 byte chunks for whole block update is really hard for me. I'm not a good programmer   
Thanks anyway

[DavidAlfa]

The're cheap spi psrams, ex. the LY68L6400 is 64Mbit.
When a file is opened for writing, you could transfer it to the RAM and work on it.
When it's closed, the full file is written back to the flash.

What about using a simple sd card?
Can be accesed in spi mode, needs some overhead to access the data, but you could forget about handling the computer side.
There're cheap sdcard-usb controllers, all you would need is detecting the USB, then setting the mcu pins interfacing the SD card in high  impedance (ex. input) and power the SD controller on.
When USB is removed, disable the SD reader and resume the control.

[iamno3]

The're cheap spi psrams, ex. the LY68L6400 is 64Mbit.
When a file is opened for writing, you could transfer it to the RAM and work on it.
When it's closed, the full file is written back to the flash.

What about using a simple sd card?
Can be accesed in spi mode, needs some overhead to access the data, but you could forget about handling the computer side.
There're cheap sdcard-usb controllers, all you would need is detecting the USB, then setting the mcu pins interfacing the SD card in high  impedance (ex. input) and power the SD controller on.
When USB is removed, disable the SD reader and resume the control.

Hello David
Thanks for the psrams reference. I will think about it if need of large RAM in future design.
My emergency panel also using SDcard and It worked fine. Renesas MCU already have library handler for USB periperal MSC using sdcard as mass storage.

[peter-h]

I am using FatFS with the Adesto serial FLASH (SPI) devices.

With a 21MHz SPI I get > 1Mbyte/sec read, and the write is limited by the 15ms "erase before write" 512 byte sector write speed, so about 30kbytes/sec write.

Windows should not be thrashing the FAT too badly. If you write say a 1MB file, it will create the dir entry, then append the data, updating the FAT at every cluster, and then at the end when the file is closed.

As stated above, local RAM cache, with a flush after a timeout, is the only solution, because you have no way of knowing what Windows (or whatever OS) will be doing next. And you can get SPI SRAMs which are physically small and fast enough for this purpose. Of course there are power-down issues with a corrupted file system but you will always have those because, like I said, there is no way to know how far Windows got with the file write. There are cunning ways to detect the file close by scanning the data and looking for a specific pattern which the Windows program inserts at the end of the file.

In the context of FLASH endurance, it is ok. Each sector can be written 50k-100k times depending on how worried you are about adjacent line disturbance limits of the FLASH.

[iamno3]

Update : I'm using 1Gbit NOR flash W25Q01 but there is some clock ticking sound on output.
If use 512 Mbit there is no ticking sound.

[ataradov]

You'd need to check with an oscilloscope what that ticking is and how it coincides with the flash access.

There are two possible options - interference from the flash access and data read failures. First one is fixed with better layout, second one with following maximum clock frequencies and other interface parameters.

[iamno3]

Dear Alex,

I've check 3,3V 5V and Opamp 15V voltage signals with osciloscope then zoomed in, the signal kinda shaking whenever mcu spi read flash data.
Check on datasheet, 1Gbit consume current more than 512Mb.
Spi clock for the flash 6MHz.
I don't know about pcb layout. Maybe my pcb have poor grounding or wrong placement of clock, mosi and miso line.
I'm sure there is no data failure when read flash. Because when connect as removable drive, compare original data on windows hardisk and data from flash both identical.

[ataradov]

It would be impossible to diagnose this without remotely. A complete review of the schematic and the board layout would be required. But realistically only experiments with the hardware would help here. "kinda shaking" is not good enough.

[iamno3]

It would be impossible to diagnose this without remotely. A complete review of the schematic and the board layout would be required. But realistically only experiments with the hardware would help here. "kinda shaking" is not good enough.

Hi Alex, I put ferrit bead between NOR VCC pin and 3.3V and there is no more ticking.
Thanks