Do SD cards handle wear leveling on their own?

[SiliconWizard]

I know this is a question the answer of which most sensibly is: "it depends".

But if anyone happens to know more about that (types of SD cards that are more likely to do this, brands, possible mentions in the SD standard, etc), that would be interesting.

[David Hess]

If the memory card has wear leveling, then it handles independently of the system that it is connected to.  Not all memory cards have wear leveling.

[woofy]

I am not aware of any SD cards that have wear leveling. The WD purple may have something, they do claim 256 TBW for their 512GB card which is around SSD range.  However that may be marketing as the application for these is in surveillance applications where they mostly cycle through the full capacity anyway. I haven't tried, but I suspect if you keep erase/writing the same sector it will fail quickly.

I'm also interested if anyone knows for sure. I did look a while back for my RPI's which keep failing with bad SD's, but never found anything definitive.

[cdev]

How can we evaluate SD cards and flash memory better? I often find myself buying flash memory from online stores and how does one buy quality, and evaluate it? I often boot SBCs off of them. How can I improve my chances of having it be reliable?

[ledtester]

Delkin produces auto-wear leveling SD cards:

https://www.delkin.com/blog/how-do-wear-leveling-sd-cards-work/

as does ATP:

https://www.atpinc.com/products/industrial-sd-cards

Both call the cards "industrial SD cards".

[SiliconWizard]

How can we evaluate SD cards and flash memory better? I often find myself buying flash memory from online stores and how does one buy quality, and evaluate it? I often boot SBCs off of them. How can I improve my chances of having it be reliable?

Evaluating, unfortunately, is kind of a lost cause. It's easy to evaluate read and write speeds in various conditions, but that's about it.
Make sure in your case that the SD cards are indeed permanently damaged, and that's it's not just a file system corruption (which is something rather common when using SD cards...)

While getting to know if a given SD card does wear leveling or not looks impossible, unless the vendor is clear about it (see above), AFAIK, eMMC *does* wear leveling. So eMMC for SBCs is not just (usually) faster, but it's also a lot more reliable. If you are using SBCs that have an option for eMMC, I really suggest consider it even if it's a bit more expensive. Unfortunately, the RPis do not, but their new compute module has eMMC AFAIR.

Otherwise yeah. Except for a very few vendors, wear leveling is like a mystery feature. It's impossible to know. I've read some claims that the WD purple cards do wear leveling, but no official statement from WD. Likewise, it's suspected that some Sandisk pro cards do, but this can't be found in the official specs. I've used Samsung microSD cards, both in the EVO and PRO series, and apart from a slight speed improvement, I am unable to tell what the PRO series offer. (For their SSDs, I think they use a different Flash technology for their PRO series, but for SD cards, I have no clue.) It's unfortunate that most vendors (at least the major ones) would not publish at least minimal datasheets for their SD cards. Most often, what you get are just commercial fact sheets.

[David Hess]

I am not aware of any SD cards that have wear leveling.

https://www.swissbit.com/en/products/nand-flash-products/sd-memory-cards/

They are also suppose to have power failure protection but I have no idea how that can be done in such a small form factor.

[RoGeorge]

Do SD cards handle wear leveling on their own?

Yes, always.

[David Hess]

Do SD cards handle wear leveling on their own?

Yes, always.

If they have it, but we know from testing that some SD cards do not implement wear leveling.

[RoGeorge]

Can you give an example of SD card without wear leveling, or how did you know they don't have it, please?

[Nominal Animal]

I don't think it is so much a question of wear leveling or not, but what kind of wear leveling is implemented.

I suspect the industrial/heavy use ones have a separate per-sector counters and indirection, whereas the consumer ones that do have wear leveling use very simple algorithms with best-guess wear leveling without spending any extra resources on it.  I do believe almost all SD cards have minimal wear leveling, perhaps some kind of linear congruential rotation of the sectors written, whose benefits vary depending on the use patterns from nil to excellent; which is why users have different experiences with the same card/series/manufacturer just by using a different hardware, file system, or OS version or firmware.  Even minimal differences in (relative) timing can affect the effectiveness of such algorithms (because it affects the order in which sectors get modified).

This is a very annoying thing when trying to run an SBC off an microSD card, because manufacturers don't describe what kind of wear leveling they use, and it's almost impossible to tell whether their expected write cycle counts (TiB or per-sector) are realistic or just marketing wank.

In my experience, if you mount the card mostly read-only, they'll survive much longer; but for anything that needs more than occasional I/O, I use eMMC or SSD.  If the SBC cannot boot off eMMC or SSD directly, then I use an SD card for the initial boot only.

Now that SBCs tend to have have USB 3, and there are cheap-but-okay USB-SATA controllers (I use a cheap Jmicron one on one of my SBCs), a small 240G SATA SSD (costs 30-40€ here) is an excellent substitute to those SD cards; and price range is similar to much smaller capacity industrial SD cards.  The SSDs take a lot more room, of course, and consume somewhat more power, but the reliability is so much better compared to what luck I've had with SD cards, it's worth it to me.

I wish there was reliable info on the reliability/wear leveling of various SD cards, though.  Sometimes one does have to use an SD card.

[David Hess]

Can you give an example of SD card without wear leveling, or how did you know they don't have it, please?

People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

[Rasz]

People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

Citation needed

[Rerouter]

Currently have a lot of old SD cards on my desk out of video records, , many of them have lost capacity from there original size, I doubt they have wear leveling, instead the device has to handle it and the diagnostics gives us information indicating its the one keeping the bad block table, many of them have many faulty blocks as that is why they where replaced and I ended up with them. though the capacity loss is the interesting part,

[cfbsoftware]

I've read some claims that the WD purple cards do wear leveling, but no official statement from WD.

FYI the following quotes related to the WD purple cards are included in this article:

https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/

Got the answer today (12 Feb 2020):

This is the statement I got back from Engineering regarding the wear leveling inquiry:

Our WD Purple MicroSD's do both static and dynamic wear-leveling. Meaning all blocks across whole address space are considered for wear leveling regardless of if and how the blocks are used. This keeps the number of program erase cycles consistent on all blocks.

So yes, these cards have proper wear-leveling according to WD. Which is nice to have confirmed directly from the manufacturer.

[Mechatrommer]

People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

Citation needed

you've got one havent you? afaik citation needed for something that exists, not something that doesnt exist...

not really that important for me, but anyway (since i already got in )...
https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/
https://electronics.stackexchange.com/questions/27619/is-it-true-that-a-sd-mmc-card-does-wear-levelling-with-its-own-controller

btw i treat SD card (similar to CF card) as expendable/simple/less reliable medium only for transit purpose.. such as transfer files captured from cameras to PC, not something serious like OS with lots of delete/write cache cycles etc... even if the card has wear levelling, the feature will be less effective when you have 50-80% allocated with permanent OS files and the rest of 20-50% spaces are forced to the death with WL.. it will die sooner anyway...

[David Hess]

I burned through a 133X 4GB Transcend Compact Flash card in a few years on my old FreeBSD router which should have had very few writes.  I replaced it with a 4GB IBM laptop hard drive which is more than 20 years old now.

One day the system stopped accepting writes and it was not clear that the Compact Flash card was the problem until I was inspired to swap in the hard drive.  After the swap, SMART data revealed that the entire allocation of spare sectors on the Flash card had been used up.

[Rasz]

People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

Citation needed

you've got one havent you? afaik citation needed for something that exists, not something that doesnt exist...

not really that important for me, but anyway (since i already got in )...
https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/
https://electronics.stackexchange.com/questions/27619/is-it-true-that-a-sd-mmc-card-does-wear-levelling-with-its-own-controller

btw i treat SD card (similar to CF card) as expendable/simple/less reliable medium only for transit purpose.. such as transfer files captured from cameras to PC, not something serious like OS with lots of delete/write cache cycles etc... even if the card has wear levelling, the feature will be less effective when you have 50-80% allocated with permanent OS files and the rest of 20-50% spaces are forced to the death with WL.. it will die sooner anyway...

You misunderstood me.

>People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

is a specific claim. Who are those people? Which cards were tested and where are those test results?

>Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

this one is even stronger and weirder. There were SSDs doing aggressive garbage collection without explicit OS support in the early days. https://www.bit-tech.net/reviews/tech/storage/windows-7-ssd-performance-and-trim/2/ But its still not clear if any went beyond exploiting transparent compression or moving data around to even wear levels. Im not familiar with _any_ hard data on SSD controllers snooping on specific filesystem metadata.

David Hess CF cards are a different story - they were IDE compatible by the spec from the beginning and almost always used simple 1:1 flash-CHS/LBA mapping with possible exception of specialized industrial ones. I personally killed few mass market CF cards in embedded applications (cheaper to replace than implement flash aware fs).

SD and USB pendrive flash controllers virtualized memory-LBA mapping from the start because at that point in time 100% flawless flash was prohibitively expensive. Im not familiar with publicly available software for SD controller management. USB is usually not a problem to obtain, Phison /Alcor etc http://www.flashdrive-repair.com

[magic]

Dealing with FAT is simple, because the only heavily written metadata (FAT tables) are at the beginning of the partition.
I have seen numerous people talking about the first few megs of an SD card being more resilient than the rest, but never any hard data.
May be an urban legend.

[David Hess]

>People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

is a specific claim. Who are those people? Which cards were tested and where are those test results?

The reports would be difficult to find now since they dated from the earliest CF and SD cards.  As James Burke said, "If it is not on the internet, then it does not exist."

Update:  Here are a bunch of examples of Compact Flash cards which do not support wear leveling.  Some of the linked technical information is still available where manufacturers like Sandisk and Kingston explicitly state so.

>Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

this one is even stronger and weirder. There were SSDs doing aggressive garbage collection without explicit OS support in the early days. https://www.bit-tech.net/reviews/tech/storage/windows-7-ssd-performance-and-trim/2/ But its still not clear if any went beyond exploiting transparent compression or moving data around to even wear levels. Im not familiar with _any_ hard data on SSD controllers snooping on specific filesystem metadata.

The way I remember it, these cards had special provisions to wear level just the FAT area, which is relatively simple to implement, so they worked deceptively well with FAT file systems, but had no wear leveling support with other filesystems like NT or any Linux filesystem.

The test of course was to write to the same sector repeatedly, which resulted in failure in short order because no wear leveling was occurring.