SD Card writing speed

[JPortici]

Hi,
I'm in the middle of my first project using micro SD cards. The card is storing logs and a settings file.
If relevant (probably is) i'm using a PIC32MX clocked at 48 MHz, SPI running at 16 MHz after card initialization. The library i'm using is the fileio included in the latest MLA (v2017_03_06).
To make it work i just had to change the SPI transfer and initialization routines to my liking.

I'm trying to make sense of the numbers, i think they are correct but i want to do a sanity check.

I'm writing the logs to a CSV file, each log entry is 130 bytes long at most (128 + CR + LF)
I measured the time taken to write a log by subtracting the value of the CoreTimer from after and before the call to an AddLog function, and divided it by (SYSCLK / 2000) to get the count in milliseconds.

The time taken varies between 10 and 11ms depending on the log type and lenght. i know that the number is rounded down, this is a quick and dirty test, it's around 11ms so let's say it's 11ms.
this means that my write speed is about 130/11*10^-3 = 11818 B/s = 11.54 kB/s

Changing the SPI frequency has little effect on the write time, infact most of the time the MCU is checking if the Card is still busy (checked with a scope)
and about 10ms to write to flash memory is in the same ballpark as other kinds of flash memory, so i suppose it's an acceptable time?

Otherwise, is there anything I can do to increase the writing speed?
Packing logs together is something i don't want to do, they will probably not be at regular intervals so the chance that i would miss one or more logs to a power down is higher

The card i'm using to test is a generic kingstone SDHC card, 8 GB class 2.

[fourtytwo42]

Yup your problem is your very small record size!

[mikerj]

You could still consider buffering the results, but write the buffer either when full or when a time limit has been reached.

[Siwastaja]

Or, redesign the software structure so that either the SD card functions run in interrupts, or everything else runs in the interrupts, so that you can do things while waiting for non-busy.

Buffering a larger packet won't help with the worst-case wait delay anyway, it actually gets even (slightly) longer, only happens more infrequently. So the question to ask is: is the average write speed number meaningful for you? Most likely, the worst case write latency matters more.

[Bruce Abbott]

The time taken varies between 10 and 11ms depending on the log type and lenght.

Is that a problem, and if so why?

[mikerj]

Or, redesign the software structure so that either the SD card functions run in interrupts, or everything else runs in the interrupts, so that you can do things while waiting for non-busy.

If that is the underlying problem for the OP then he doesn't need to move all the SD handling to interrupts.  The busy status should not need to be constantly polled until a write is completed, instead check it prior to a write.  If it's not busy then do the write immediately, if it is busy then stick the data into a buffer and set a timer to initiate the next write.

[JPortici]

Thanks @fourtytwo64 for confirming my suspicions.

I am not concerned that i'm wasting 10ms doing nothing, as all the firmware is made of a number of state machines, the data to be processed is stored in buffers, i can waste 10ms from time to time.
Yes, from time to time the firmware will block for around 10ms but the main state machine has a time step of 25ms. there shouldn't be any problem if once in a blue moon one step is processed later

The Filesystem is the only part of my firmware that has blocking code, it is also the only part that was not written by me.
I know it's not ideal but for now i will keep it like this, until i have the courage to modify the library or i look for another one

[@rt]

How did you determine your SPI clock speed?

I don’t know how fast any particular card will be,
but the way I found out was adjusting the prescaler
in the library until FSFInit gives a failed result,
and then back off the prescaler from there.

Since I don’t do much writing, I can’t really say what is slow or fast.

[SiliconWizard]

There's nothing much you can do apart from increasing the buffer size (and thus the risk of data loss), so this is a compromise.

Or switch to a different technology, but as I reckon, this may not be practical or cost-effective compared to SD cards if you deal with great amounts of log data.

So as a mitigating solution, I would suggest the following: use an intermediate, non-volatile memory to buffer your data. A very effective solution would be to use an FRAM chip, such as a 256Kbits SPI FRAM, and log your data in it. Then when the FRAM chip is half full, just flush its content to the SD card while you can still interleave log writes to it. Ideally, if you have a DMA channel available, the flushing can be in with DMA in chunks big enough that this would block your CPU only for very short periods of time. If the double buffering scheme with only one chip seems intimidating or unpractical, you could also use 2 FRAM chips and alternate. FRAM have very fast write cycles (no wait state) and have been proven as robust, if not more, than Flash. (Of course you could only use this as intermediate buffers since they only come in low capacity.)

[technix]

I actually detected another problem in this use mode, which can extend the write time and potentially ruin the NAND in the microSD card a lot faster than you think: every time you write down a line of log, it might need to perform two NAND page erases: erase the page where your new data is to be stored, and erase the page where the directory listing is stored to update the file size. Each erase would have to involve a read-modify-write cycle: put 4kB or so into the microSD's internal RAM, erase the data, modify the data in that RAM, and write everything back.

I would suggest using a local NOR-based write cache like a 25-series Flash chip. When each log item is generated it is written to the cache chip first. When the NOR chip is filled or when the user requested to do so, perform a bulk write to the microSD with the data from the NOR chip (which doesn't involve a lot of read-modify-write cycles since most of the data is that big multi-page bulk write) and bulk erase the entire NOR write cache chip afterwards.

[mikeselectricstuff]

Whan I was doing the SD card stuff for the hackaday badge (PIC32MX170 using Microchip's library with a few tweaks to use 32-bit transfers for speed), we found huge differences in write speeds between different types of card, like factors of over 3x
 
We found that smaller cards 128M and most 256M were substantially faster, larger cards 1G and above were always substantially slower. From memory I think the rates ranged from around  100 to 350KBytes/sec writing fairly large blocks ( tens of K) 

My gut feeling, not bourne out by any actual tests or detailed understanding of the low-level protocols, is that the Microchip library always writes in 512 byte blocks, and the larger cards have a larger native block size, and are internally doing some kind of read/modify/write cycle, or block reassign+erase when writing 512 byte blocks in SPI mode.

Read speeds were also rather variable between card types, but only by about 2x - I put this down to them not caring about efficiently implementing SPI mode.
 

[helius]

Yes, anything that makes modifications to data smaller than the page size will cause write amplification. The reason for Flash File Systems is to minimize erase cycles by using clever tricks to store the data (the primary one is the log structure: data is always appended instead of modified, and the host does a scan when mounting the flash to discover the position of the latest pointers in the log structure).
Another important aspect is the TRIM command that tells the drive "I no longer need this block, recycle it". Without a way to release blocks from the used pool to the free pool, the wear-leveling algorithm in the drive must steadily increase the number of erase cycles it performs per block written.

[ucanel]

I read all answers and all of them has good information for the question,
no need to repeat them.
I wrote 3 library type codes for sd card and FAT system and 2 of them was in asm.

If it works for you there is a little bit crude solution for the problem:
For only once create enough file for your logs lets say 100 or 1000 or X pieces of file,
and all of the files will be created with equal sizes,
and the file size need to be selected equal to the Cluster size,
(whatever you do it is always equal or multiples of the cluster size anyway,
you may recognize it from windows> right click> properties > size on disk vs size)
and than you will modify the library for getting the first file's address, cluster size,
you will reach Xth file with first file address + X * Cluster size,
you will also find the raw write function it is fundamentally CMD24 and CMD25 (0x58, 0x59)
with the file start address and CMD24 or CMD25 you may write at the speed of native sd spi write speed limits,
if you need or in case you interested the speed is stated at the SD card datasheet and I could find it for you.

[technix]

I read all answers and all of them has good information for the question,
no need to repeat them.
I wrote 3 library type codes for sd card and FAT system and 2 of them was in asm.

If it works for you there is a little bit crude solution for the problem:
For only once create enough file for your logs lets say 100 or 1000 or X pieces of file,
and all of the files will be created with equal sizes,
and the file size need to be selected equal to the Cluster size,
(whatever you do it is always equal or multiples of the cluster size anyway,
you may recognize it from windows> right click> properties > size on disk vs size)
and than you will modify the library for getting the first file's address, cluster size,
you will reach Xth file with first file address + X * Cluster size,
you will also find the raw write function it is fundamentally CMD24 and CMD25 (0x58, 0x59)
with the file start address and CMD24 or CMD25 you may write at the speed of native sd spi write speed limits,
if you need or in case you interested the speed is stated at the SD card datasheet and I could find it for you.

I am still standing by the write cache idea. Don’t write to the SD card just yet when logs are being generated since those small writes will result in write amplification and slow down access greatly. Let the log build up in the write cache device (NOR Flash or FRAM are the best here, since they don’t suffer from write amplification, and they allow byte-scale writing.) to the point it is big enough to almost or exactly fill up to an integral multiple of a block. Then perform a big write dumping the entire contents of the write cache device into a single file on the SD card. Now you have avoided the dreaded small write access.

Actually depend on how much log your device may generate, a higher capacity SPI NOR like W25Q128 (16MiB) by itself might be adequate. Then instead of adding that SD card slot you need a data uplink so the contents of the Flash chip can be uploaded to a host machine.

[JPortici]

My gut feeling, not bourne out by any actual tests or detailed understanding of the low-level protocols, is that the Microchip library always writes in 512 byte blocks, and the larger cards have a larger native block size, and are internally doing some kind of read/modify/write cycle, or block reassign+erase when writing 512 byte blocks in SPI mode.

Read speeds were also rather variable between card types, but only by about 2x - I put this down to them not caring about efficiently implementing SPI mode.
 

aren't you also limited by the fact that the <microchip> fileio library uses 512 bytes clusters? (please correct me if i'm wrong, i'm still new to this whole sd/filesystem thing)

i was concerned about card wearout as well, but i calculated that a worst case scenario would be 8 hours straight of logging (but not every day), an average of one entry every 100ms so 128*10*60*60*8 = 35MB of data which wit in a 512 MBit flash memory. I don't really want to go BGA..

I gave a thought about a small eeprom or simillar to buffer the data, but the board is going to be very small (40x18mm), components on both sides (and one side is mostly the MCU and BLE SOC) so everything i can remove i will remove. And given the current price of 2/4 and even 8 GByte uSD cards all things considered i think that i can handle having to replace it every few years..

[mikeselectricstuff]

Do you need the logs to be ASCII CSV ? You could reduce the data by a significant factor by using a binary format

[JPortici]

I know, but we want them to be in a human readable format

[frozenfrogz]

This is just somewhat related as it deals with the SDfat library on Arduino but maybe it helps.
http://forum.arduino.cc/index.php?topic=49649

The main issue that is addressed here is that the wait for data transmission complete flush() of the serial buffer slows down the writing process. The recommendation is to flush after 100 write cycles instead of every cycle.

[ucanel]

I am still standing by the write cache idea. Don’t write to the SD card just yet when logs are being generated since those small writes will result in write amplification and slow down access greatly.
....

You are right and your solution is good enough if you have extra space and money.
Actually there is another solution hidden in standard way of sd communication,
yes you need to write or read at least 512 byte = 1 block (generally)
but you don't have to read or write without any stalls,
you could give sd card a clock with spi etc. get one byte (or even a bit) and then wait 10 ms
and then get  or write another byte from or to it,
i tried this method with 1 hour intervals it does not matter, works great.
In this case you do not need any extra buffer ram/flash
sd card itself works as if one.

[C]

I know, but we want them to be in a human readable format

Writing human readable format does take software time & more space.
A simple function read binary & output human could be nice.
You would gain increased speed during test & able to log more data in same space.
If the SD card stays in device, you could have a task that runs in free cpu time that outputs human to connected thing or creates human file from binary.
Or
You could have a function run at end of test to create human file.

[iMo]

i was concerned about card wearout as well, but i calculated that a worst case scenario would be 8 hours straight of logging (but not every day),

All the sdcards today use "wear-leveling" (and they do maybe other internal housekeeping tasks as well), which always runs in background (therefore there are the frequent random outages, lasting from a few ms up to 200ms, where the sdcard is not accessible(!!) ).

Thus if you have got say a 4GB sdcard and you write 1MB/day it will last for 5000*4000MB/1MB days
PS: 5000 is a typical number of writes to a flash cell (until it wears-out) with a 4GB card , usually it lasts more writes..

PS: Below some rd/wr speed benchmarks with various SDcards (vendor/size), ran on pic32MX (retrobsd) and pic32MZ (litebsd) under Unix, mind the SPI clock with each test..
http://www.retrobsd.org/wiki/doku.php/doc/sd-benchmark#microsd_cardsretrobsd
http://www.retrobsd.org/wiki/doku.php/doc/sd-benchmark#microsd_cardslitebsd
https://github.com/sergev/LiteBSD/wiki/SD-Card-Benchmark

[@rt]

For a bunch of small files, write speed is the least of your worries.
MDDFS takes longer to open a file the more files exist in a directory.
It would be much faster to open and append a pair of identical files.

Compare the time it takes to create the first, and 1000th file in a directory.

[SimonR]

Whan I was doing the SD card stuff for the hackaday badge (PIC32MX170 using Microchip's library with a few tweaks to use 32-bit transfers for speed), we found huge differences in write speeds between different types of card, like factors of over 3x
 
We found that smaller cards 128M and most 256M were substantially faster, larger cards 1G and above were always substantially slower. From memory I think the rates ranged from around  100 to 350KBytes/sec writing fairly large blocks ( tens of K) 

My gut feeling, not bourne out by any actual tests or detailed understanding of the low-level protocols, is that the Microchip library always writes in 512 byte blocks, and the larger cards have a larger native block size, and are internally doing some kind of read/modify/write cycle, or block reassign+erase when writing 512 byte blocks in SPI mode.

Read speeds were also rather variable between card types, but only by about 2x - I put this down to them not caring about efficiently implementing SPI mode.
 

Mike

You could well be correct about your suspicions I had similar variable results with Compact Flash cards.

For SD cards there was a big change in the spec to support big capacities around 2007. At the time the biggest card that could be supported was about 2G (I think).
In the original SD spec the card address was in bytes but to support bigger sizes the card address changed to be block addresses. This actually allows for up to 4Tbyte but the spec only says up to 32GB, I don't know why.
All cards power up in Byte Address SPI mode to be backward compatible and then need an extra step in the initialisation to put then in Block Mode or parallel bus mode.

Possibly around this time the internal architecture started changing as well resulting in all the variations of speed of access.

My own findings with compact flash might be helpful.
I don't support the high speed UDMA mode, I only use the basic start up mode but even so I have seen huge differences across manufacturers in my application. And even big changes from one version to the next of the SAME card.

For large writes they are mostly consistent around 6 Mbyte/s but one or two won't go faster then 3 Mbyte/S.

For small writes things are really variable. This is typically around 1 Mbyte/S but I have some cards that only do 100Kbytes/s.
I had 1 card that only did 10Kbytes/S  .We sent those back to the manufacturer.

If you rely on the card speed it is worth evaluating lots of different cards to find one that meets your requirements but be aware that if you are buying commercial cards you may not get the same firmware next time you order them.
If you buy industrial cards they may cost more but you will get a part number to order against and you are guaranteed the same firmware for the life of the product.
Product lifetime seems to be about 18 months so you end up having to re-qualify every 18 months but at least you can plan for that.

[@rt]

For SDSC & SDHC cards, there are clear differences in their registers,
and the negotiation for initialising them.
It may be only V2 cards (that are SDHC) that may allow lower voltage.

SDHC & SDXC are confusing. The only difference I’ve found between them are that
no SDHC Card is greater than 32Gb, and every SDXC Card is greater than 32Gb,
SDXC cards advertise faster transfer rates, SDXC comes pre-formatted exFAT.
These are all rather arbitrary though.

[SimonR]

That's interesting about SDXC having different registers.

When I last used SD memory cards was 10 years ago and SDXC didn't exist. They were still arguing about the standard for high speed cards at the time.
The SDHC standard definitely said that 32GB was the limit even though it was technically capable of going bigger.
They were also talking about it requiring exFat but I don't remember if the SDHC cards were being shipped with exFat or FAT16/32. I have a vague recollection that they were still shipping as FAT32.

I don't remember that clearly as I only used memory cards to get the driver working. I was actually developing an SDIO peripheral.