Author Topic: Do SD cards handle wear leveling on their own?  (Read 3954 times)

0 Members and 1 Guest are viewing this topic.

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 9516
  • Country: fr
Do SD cards handle wear leveling on their own?
« on: November 19, 2021, 05:01:22 pm »
I know this is a question the answer of which most sensibly is: "it depends".

But if anyone happens to know more about that (types of SD cards that are more likely to do this, brands, possible mentions in the SD standard, etc), that would be interesting.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #1 on: November 19, 2021, 11:33:29 pm »
If the memory card has wear leveling, then it handles independently of the system that it is connected to.  Not all memory cards have wear leveling.
 

Offline woofy

  • Regular Contributor
  • *
  • Posts: 167
  • Country: gb
    • Woofys Place
Re: Do SD cards handle wear leveling on their own?
« Reply #2 on: November 19, 2021, 11:34:58 pm »
I am not aware of any SD cards that have wear leveling. The WD purple may have something, they do claim 256 TBW for their 512GB card which is around SSD range.  However that may be marketing as the application for these is in surveillance applications where they mostly cycle through the full capacity anyway. I haven't tried, but I suspect if you keep erase/writing the same sector it will fail quickly.

I'm also interested if anyone knows for sure. I did look a while back for my RPI's which keep failing with bad SD's, but never found anything definitive.


Offline cdev

  • Super Contributor
  • ***
  • !
  • Posts: 7350
  • Country: 00
Re: Do SD cards handle wear leveling on their own?
« Reply #3 on: November 19, 2021, 11:41:14 pm »
How can we evaluate SD cards and flash memory better? I often find myself buying flash memory from online stores and how does one buy quality, and evaluate it? I often boot SBCs off of them. How can I improve my chances of having it be reliable?
"What the large print giveth, the small print taketh away."
 

Online ledtester

  • Super Contributor
  • ***
  • Posts: 2297
  • Country: us
Re: Do SD cards handle wear leveling on their own?
« Reply #4 on: November 20, 2021, 12:40:02 am »
Delkin produces auto-wear leveling SD cards:

https://www.delkin.com/blog/how-do-wear-leveling-sd-cards-work/

as does ATP:

https://www.atpinc.com/products/industrial-sd-cards

Both call the cards "industrial SD cards".
 
The following users thanked this post: fourtytwo42, SiliconWizard, woofy

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 9516
  • Country: fr
Re: Do SD cards handle wear leveling on their own?
« Reply #5 on: November 20, 2021, 01:27:42 am »
How can we evaluate SD cards and flash memory better? I often find myself buying flash memory from online stores and how does one buy quality, and evaluate it? I often boot SBCs off of them. How can I improve my chances of having it be reliable?

Evaluating, unfortunately, is kind of a lost cause. It's easy to evaluate read and write speeds in various conditions, but that's about it.
Make sure in your case that the SD cards are indeed permanently damaged, and that's it's not just a file system corruption (which is something rather common when using SD cards...)

While getting to know if a given SD card does wear leveling or not looks impossible, unless the vendor is clear about it (see above), AFAIK, eMMC *does* wear leveling. So eMMC for SBCs is not just (usually) faster, but it's also a lot more reliable. If you are using SBCs that have an option for eMMC, I really suggest consider it even if it's a bit more expensive. Unfortunately, the RPis do not, but their new compute module has eMMC AFAIR.

Otherwise yeah. Except for a very few vendors, wear leveling is like a mystery feature. It's impossible to know. I've read some claims that the WD purple cards do wear leveling, but no official statement from WD. Likewise, it's suspected that some Sandisk pro cards do, but this can't be found in the official specs. I've used Samsung microSD cards, both in the EVO and PRO series, and apart from a slight speed improvement, I am unable to tell what the PRO series offer. (For their SSDs, I think they use a different Flash technology for their PRO series, but for SD cards, I have no clue.) It's unfortunate that most vendors (at least the major ones) would not publish at least minimal datasheets for their SD cards. Most often, what you get are just commercial fact sheets.

 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #6 on: November 20, 2021, 02:06:04 am »
I am not aware of any SD cards that have wear leveling.

https://www.swissbit.com/en/products/nand-flash-products/sd-memory-cards/

They are also suppose to have power failure protection but I have no idea how that can be done in such a small form factor.
 
The following users thanked this post: fourtytwo42, SiliconWizard, woofy

Offline RoGeorge

  • Super Contributor
  • ***
  • Posts: 4004
  • Country: ro
Re: Do SD cards handle wear leveling on their own?
« Reply #7 on: November 20, 2021, 02:11:34 am »
Do SD cards handle wear leveling on their own?

Yes, always.

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #8 on: November 20, 2021, 02:46:47 am »
Do SD cards handle wear leveling on their own?

Yes, always.

If they have it, but we know from testing that some SD cards do not implement wear leveling.
 

Offline RoGeorge

  • Super Contributor
  • ***
  • Posts: 4004
  • Country: ro
Re: Do SD cards handle wear leveling on their own?
« Reply #9 on: November 20, 2021, 03:15:46 am »
Can you give an example of SD card without wear leveling, or how did you know they don't have it, please?

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 3699
  • Country: fi
    • My home page and email address
Re: Do SD cards handle wear leveling on their own?
« Reply #10 on: November 20, 2021, 06:06:31 pm »
I don't think it is so much a question of wear leveling or not, but what kind of wear leveling is implemented.

I suspect the industrial/heavy use ones have a separate per-sector counters and indirection, whereas the consumer ones that do have wear leveling use very simple algorithms with best-guess wear leveling without spending any extra resources on it.  I do believe almost all SD cards have minimal wear leveling, perhaps some kind of linear congruential rotation of the sectors written, whose benefits vary depending on the use patterns from nil to excellent; which is why users have different experiences with the same card/series/manufacturer just by using a different hardware, file system, or OS version or firmware.  Even minimal differences in (relative) timing can affect the effectiveness of such algorithms (because it affects the order in which sectors get modified).

This is a very annoying thing when trying to run an SBC off an microSD card, because manufacturers don't describe what kind of wear leveling they use, and it's almost impossible to tell whether their expected write cycle counts (TiB or per-sector) are realistic or just marketing wank.

In my experience, if you mount the card mostly read-only, they'll survive much longer; but for anything that needs more than occasional I/O, I use eMMC or SSD.  If the SBC cannot boot off eMMC or SSD directly, then I use an SD card for the initial boot only.

Now that SBCs tend to have have USB 3, and there are cheap-but-okay USB-SATA controllers (I use a cheap Jmicron one on one of my SBCs), a small 240G SATA SSD (costs 30-40€ here) is an excellent substitute to those SD cards; and price range is similar to much smaller capacity industrial SD cards.  The SSDs take a lot more room, of course, and consume somewhat more power, but the reliability is so much better compared to what luck I've had with SD cards, it's worth it to me.

I wish there was reliable info on the reliability/wear leveling of various SD cards, though.  Sometimes one does have to use an SD card.
 
The following users thanked this post: simba15, DiTBho

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #11 on: November 20, 2021, 08:31:28 pm »
Can you give an example of SD card without wear leveling, or how did you know they don't have it, please?

People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.
 
The following users thanked this post: fourtytwo42

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2571
  • Country: 00
    • My random blog.
Re: Do SD cards handle wear leveling on their own?
« Reply #12 on: December 05, 2021, 09:45:57 am »
People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

Citation needed
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline Rerouter

  • Super Contributor
  • ***
  • Posts: 4682
  • Country: au
  • Question Everything... Except This Statement
Re: Do SD cards handle wear leveling on their own?
« Reply #13 on: December 05, 2021, 09:55:06 am »
Currently have a lot of old SD cards on my desk out of video records, , many of them have lost capacity from there original size, I doubt they have wear leveling, instead the device has to handle it and the diagnostics gives us information indicating its the one keeping the bad block table, many of them have many faulty blocks as that is why they where replaced and I ended up with them. though the capacity loss is the interesting part,
 

Online cfbsoftware

  • Regular Contributor
  • *
  • Posts: 52
  • Country: au
    • Astrobe: Oberon IDE for Cortex-M and FPGA Development
Re: Do SD cards handle wear leveling on their own?
« Reply #14 on: December 05, 2021, 09:50:48 pm »
I've read some claims that the WD purple cards do wear leveling, but no official statement from WD.

FYI the following quotes related to the WD purple cards are included in this article:

https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/

Quote
Got the answer today (12 Feb 2020):

Quote
This is the statement I got back from Engineering regarding the wear leveling inquiry:

Our WD Purple MicroSD's do both static and dynamic wear-leveling. Meaning all blocks across whole address space are considered for wear leveling regardless of if and how the blocks are used. This keeps the number of program erase cycles consistent on all blocks.

So yes, these cards have proper wear-leveling according to WD. Which is nice to have confirmed directly from the manufacturer.
Chris Burrows
CFB Software
https://www.astrobe.com
 
The following users thanked this post: SiliconWizard

Offline Mechatrommer

  • Super Contributor
  • ***
  • Posts: 10520
  • Country: my
  • reassessing directives...
Re: Do SD cards handle wear leveling on their own?
« Reply #15 on: December 05, 2021, 11:13:37 pm »
People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.
Citation needed
you've got one havent you? afaik citation needed for something that exists, not something that doesnt exist...

not really that important for me, but anyway (since i already got in :palm:)...
https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/
https://electronics.stackexchange.com/questions/27619/is-it-true-that-a-sd-mmc-card-does-wear-levelling-with-its-own-controller

btw i treat SD card (similar to CF card) as expendable/simple/less reliable medium only for transit purpose.. such as transfer files captured from cameras to PC, not something serious like OS with lots of delete/write cache cycles etc... even if the card has wear levelling, the feature will be less effective when you have 50-80% allocated with permanent OS files and the rest of 20-50% spaces are forced to the death with WL.. it will die sooner anyway...
Nature: Evolution and the Illusion of Randomness (Stephen L. Talbott): Its now indisputable that... organisms “expertise” contextualizes its genome, and its nonsense to say that these powers are under the control of the genome being contextualized - Barbara McClintock
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #16 on: December 06, 2021, 12:11:04 am »
I burned through a 133X 4GB Transcend Compact Flash card in a few years on my old FreeBSD router which should have had very few writes.  I replaced it with a 4GB IBM laptop hard drive which is more than 20 years old now.

One day the system stopped accepting writes and it was not clear that the Compact Flash card was the problem until I was inspired to swap in the hard drive.  After the swap, SMART data revealed that the entire allocation of spare sectors on the Flash card had been used up.
 

Offline Rasz

  • Super Contributor
  • ***
  • Posts: 2571
  • Country: 00
    • My random blog.
Re: Do SD cards handle wear leveling on their own?
« Reply #17 on: December 06, 2021, 11:02:10 am »
People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.
Citation needed
you've got one havent you? afaik citation needed for something that exists, not something that doesnt exist...

not really that important for me, but anyway (since i already got in :palm:)...
https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/
https://electronics.stackexchange.com/questions/27619/is-it-true-that-a-sd-mmc-card-does-wear-levelling-with-its-own-controller

btw i treat SD card (similar to CF card) as expendable/simple/less reliable medium only for transit purpose.. such as transfer files captured from cameras to PC, not something serious like OS with lots of delete/write cache cycles etc... even if the card has wear levelling, the feature will be less effective when you have 50-80% allocated with permanent OS files and the rest of 20-50% spaces are forced to the death with WL.. it will die sooner anyway...

You misunderstood me.

>People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

is a specific claim. Who are those people? Which cards were tested and where are those test results?

>Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

this one is even stronger and weirder. There were SSDs doing aggressive garbage collection without explicit OS support in the early days. https://www.bit-tech.net/reviews/tech/storage/windows-7-ssd-performance-and-trim/2/ But its still not clear if any went beyond exploiting transparent compression or moving data around to even wear levels. Im not familiar with _any_ hard data on SSD controllers snooping on specific filesystem metadata.

David Hess CF cards are a different story - they were IDE compatible by the spec from the beginning and almost always used simple 1:1 flash-CHS/LBA mapping with possible exception of specialized industrial ones. I personally killed few mass market CF cards in embedded applications (cheaper to replace than implement flash aware fs).

SD and USB pendrive flash controllers virtualized memory-LBA mapping from the start because at that point in time 100% flawless flash was prohibitively expensive. Im not familiar with publicly available software for SD controller management. USB is usually not a problem to obtain, Phison /Alcor etc http://www.flashdrive-repair.com
Who logs in to gdm? Not I, said the duck.
My fireplace is on fire, but in all the wrong places.
 

Offline magic

  • Super Contributor
  • ***
  • Posts: 4663
  • Country: pl
Re: Do SD cards handle wear leveling on their own?
« Reply #18 on: December 06, 2021, 02:27:56 pm »
Dealing with FAT is simple, because the only heavily written metadata (FAT tables) are at the beginning of the partition.
I have seen numerous people talking about the first few megs of an SD card being more resilient than the rest, but never any hard data.
May be an urban legend.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 14428
  • Country: us
  • DavidH
Re: Do SD cards handle wear leveling on their own?
« Reply #19 on: December 06, 2021, 09:24:18 pm »
>People have run tests by writing data to the same sector repeatedly, and the SD cards without wear leveling failed in short order consistent with the write endurance of a single Flash page.

is a specific claim. Who are those people? Which cards were tested and where are those test results?

The reports would be difficult to find now since they dated from the earliest CF and SD cards.  As James Burke said, "If it is not on the internet, then it does not exist."

Update:  Here are a bunch of examples of Compact Flash cards which do not support wear leveling.  Some of the linked technical information is still available where manufacturers like Sandisk and Kingston explicitly state so.

Quote
>Some cards were also found to rely on the file system structure. so for instance they might only wear level the FAT area, and do nothing at all with NTFS or whatever.

this one is even stronger and weirder. There were SSDs doing aggressive garbage collection without explicit OS support in the early days. https://www.bit-tech.net/reviews/tech/storage/windows-7-ssd-performance-and-trim/2/ But its still not clear if any went beyond exploiting transparent compression or moving data around to even wear levels. Im not familiar with _any_ hard data on SSD controllers snooping on specific filesystem metadata.

The way I remember it, these cards had special provisions to wear level just the FAT area, which is relatively simple to implement, so they worked deceptively well with FAT file systems, but had no wear leveling support with other filesystems like NT or any Linux filesystem.

The test of course was to write to the same sector repeatedly, which resulted in failure in short order because no wear leveling was occurring.
« Last Edit: December 07, 2021, 09:24:31 am by David Hess »
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf