Explaining large disparities in NVME SSD TBW endurances

[Lenclume]

Hello, I'm in the process of specing an SSD for my next desktop PC. The said SSD is going to be the boot, and only retentive memory in it.

Comparing a few recent M2 NVME drives yields fairly consistent results in terms of read/write speed, around 3000MB/s. However, TBW endurance values vary wildy for one brand to the next.

SSD lines compared at 2TB

Brand	Sabrent	WD	Samsung	Corsair	Viper/Patriot Memory
Model	SB-RKTQ-2TB	WDS200T3X0C	MZ-V7S2T0B/AM	CSSD-F1920GBMP510	VPN100-2TBM28H
Random 4KB IOPs	255K R, 670K W	480K R, 550K W	620K R, 560K W	485K R, 530K W	700K R, 700K W
Sequential speed (MB/s, ATTO/CDM averaged)	3200R, 3000W	3400R, 2900W	3500R, 3300W	3480R, 2700W	3350R, 3150W
Cost (CAD$ on PCPartpicker)	370$	488$	648$	427$	415$
TBW Endurance	530	1200	1200	3120	3115

First reflex is to to jump to the high-endurance and fast Viper/Patriot drive, however WD and Samsung are both more reputed in that domain, so my questions are:

Where's the catch? What causes high/low TBWs?

Sources:
https://ca.pcpartpicker.com/products/internal-hard-drive/#D=1&i=83,85&f=122110,122080&X=0,158221&A=1800000000000,16000000000000&sort=price&page=1
https://www.sabrent.com/rocket-q/
https://www.westerndigital.com/products/internal-drives/wd-black-sn750-nvme-ssd
https://www.samsung.com/us/computing/memory-storage/solid-state-drives/ssd-970-evo-plus-nvme-m-2-2-tb-mz-v7s2t0b-am/
https://www.corsair.com/ca/en//Categories/Products/Storage/M-2-SSDs/Force-Series-MP510/p/CSSD-F1920GBMP510#tab-tech-specs
https://assets.website-files.com/5cdb2ee0b102f96c3906500f/5dd5bdd6b042022aa8268784_Viper_VPN100_Product_Sheet_2TB.pdf

[Wuerstchenhund]

Hello, I'm in the process of specing an SSD for my next desktop PC. The said SSD is going to be the boot, and only retentive memory in it.

Comparing a few recent M2 NVME drives yields fairly consistent results in terms of read/write speed, around 3000MB/s. However, TBW endurance values vary wildy for one brand to the next.

SSD lines compared at 2TB

Brand	Sabrent	WD	Samsung	Corsair	Viper/Patriot Memory
Model	SB-RKTQ-2TB	WDS200T3X0C	MZ-V7S2T0B/AM	CSSD-F1920GBMP510	VPN100-2TBM28H
Random 4KB IOPs	255K R, 670K W	480K R, 550K W	620K R, 560K W	485K R, 530K W	700K R, 700K W
Sequential speed (MB/s, ATTO/CDM averaged)	3200R, 3000W	3400R, 2900W	3500R, 3300W	3480R, 2700W	3350R, 3150W
Cost (CAD$ on PCPartpicker)	370$	488$	648$	427$	415$
TBW Endurance	530	1200	1200	3120	3115

First reflex is to to jump to the high-endurance and fast Viper/Patriot drive, however WD and Samsung are both more reputed in that domain, so my questions are:

I'm not sure I would consider WDC "reputed" when it comes to SSDs, and their reputation with spinning rust is somewhat questionable (of all the hard disk manufacturers that are still in existence, WDC has given us by far the most trouble over the last 20 years or so).

As to Samsung, well, they are decent, although their track history when it comes to SSDs isn't exactly spotless either.

Where's the catch? What causes high/low TBWs?

There's no catch. The TBW is a function of the memory cell endurance of the flash memory that is used together with the amount of spare cells provided (overprovisioning). It's as simple as that.

Most consumer drives are below 1PBW, while enterprise drives are generally in the multiple PWB range.

Personally, I had quite a few consumer grade SSDs (mostly Samsung 840/850 EVO and intel 700s) which reached their limits quicker than estimated so now I'm only buying SSDs which have an endurance rating on the PB range (mostly Micron 9200/9300 and 5100/5200 Series these days).

[David Hess]

Personally, I had quite a few consumer grade SSDs (mostly Samsung 840/850 EVO and intel 700s) which reached their limits quicker than estimated so now I'm only buying SSDs which have an endurance rating on the PB range (mostly Micron 9200/9300 and 5100/5200 Series these days).

My consumer grade SSDs are also wearing out faster than estimated.  Like retention time which is hardly ever specified, I suspect endurance is routinely exaggerated.

[rrinker]

 Guess it depends on what you're doing. My first SSD from I'm not sure anymore how many years ago (more than 7, for sure, it was long before I moved to this house) is still going strong, although being small, it's relegated to being a secondary data drive. A larger one from my previous system is now the data drive in my current system I just build, the primary being an NVME m.2 drive, and SMART data shows tons of life left in it - this used to be the C drive in previous machine, so with everything that entails running Windows 7 and then Windows 10.

TO be failing - must be some pretty high level workloads, lots of video editing or some other sort of large data set manipulation?

As far as ssd manufacturer - there are limited suppliers of the flash memory, so it comes down at least as much to whose memory is used in a particular drive, more so than who the vendor is. There are also relatively few controller manufactures, although firmware may be customized by specific vendors.

[Lenclume]

Thanks for all the replies,

There is a consensus that consumer-grade SSDs will wear out before their due date (depending on what it's doing). Going by the numbers, I will be getting the VPN100 (3115 TBW). Crossing fingers that the manufacturers' numbers are trustworthy.

At best they would come from a standardized reproducible test , at worst they're pulling those out of their hats with a good dose of inflation.

This article also suggests that choosing a motherboard with a heatsink strip on the PCB M.2 slot improves durability: https://www.maketecheasier.com/does-nvme-ssd-need-heatsink/

[wraper]

SB-RKTQ-2TB uses QLC Flash.

[wraper]

Corsair and Patriot use Toshiba 64-layer BiCS3 TLC, WD uses Sandisk 64L TLC which is likely exactly the same as Toshiba as they make their NAND together. And Samsung - Samsung 64L V-NAND TLC.
Based on this Corsair and Patriot seem to be more optimistic with their TBW as they supposedly use the same NAND as other SSD with much lower TBW.
EDIT: BTW I suggest looking at ASX8200PNP-2TT-C. Adata actually is larger SSD manufacturer with longer expertise than most on the list. Uses 64L TLC Micron/Intel NAND. And costs about the same as crappy Sabrent QLC, 5-year warranty.
https://www.tomshardware.com/reviews/adata-xpg-sx8200-pro-ssd,5955-4.html

[wraper]

I'm not sure I would consider WDC "reputed" when it comes to SSDs, and their reputation with spinning rust is somewhat questionable (of all the hard disk manufacturers that are still in existence, WDC has given us by far the most trouble over the last 20 years or so).

Seagate is constantly so far ahead at producing junk, there is no competition.

[bdunham7]

Thanks for all the replies,

There is a consensus that consumer-grade SSDs will wear out before their due date (depending on what it's doing). Going by the numbers, I will be getting the VPN100 (3115 TBW). Crossing fingers that the manufacturers' numbers are trustworthy.

At best they would come from a standardized reproducible test , at worst they're pulling those out of their hats with a good dose of inflation.

This article also suggests that choosing a motherboard with a heatsink strip on the PCB M.2 slot improves durability: https://www.maketecheasier.com/does-nvme-ssd-need-heatsink/

I have zero faith in the manufacturer TBW spec meaning much in real life.  It's sort of like MTBF.  Not a useful spec unless you understand all of the details.  FWIW, I won't use anything other than Samsung based on my own personal experience, although I have never used the purportedly problematic 840EVO.  The only other manufacturer that I have not had a failure with is Sandisk, and that's a sample size of two.  I think you are either vastly overestimating the amount of written data in your use case or you aren't configuring things optimally.  The computer I am typing on now has a Samsung 850EVO 1TB that is a few years old now and is in its second computer after the first met with an accident.  It is a laptop that is left on most of the time and used all day; it is the primary drive with a secondary for video editing and so on, and I have page/swap file stuff disabled.  My total TBW to date is....eleven (11).

In short I think the difference between a spec of 1200TBW and 3100TBW is meaningless in your case because you ought not ever get anywhere near those numbers, the specs are from two different manufacturers so only they know what they mean (it's called aggressive specmanship) and if you do have a failure, it will likely be due to some other reason. 

[David Hess]

Guess it depends on what you're doing. My first SSD from I'm not sure anymore how many years ago (more than 7, for sure, it was long before I moved to this house) is still going strong, although being small, it's relegated to being a secondary data drive. A larger one from my previous system is now the data drive in my current system I just build, the primary being an NVME m.2 drive, and SMART data shows tons of life left in it - this used to be the C drive in previous machine, so with everything that entails running Windows 7 and then Windows 10.

TO be failing - must be some pretty high level workloads, lots of video editing or some other sort of large data set manipulation?

I have been using them for page and temporary files where their performance is an advantage.  The scale is different but for example, my router runs BSD and when I changed from m0n0wall to pfsense, it burned up the compact flash card in less than a year.

[Lenclume]

Corsair and Patriot use Toshiba 64-layer BiCS3 TLC, WD uses Sandisk 64L TLC which is likely exactly the same as Toshiba as they make their NAND together. And Samsung - Samsung 64L V-NAND TLC.
Based on this Corsair and Patriot seem to be more optimistic with their TBW as they supposedly use the same NAND as other SSD with much lower TBW.
EDIT: BTW I suggest looking at ASX8200PNP-2TT-C. Adata actually is larger SSD manufacturer with longer expertise than most on the list. Uses 64L TLC Micron/Intel NAND. And costs about the same as crappy Sabrent QLC, 5-year warranty.
https://www.tomshardware.com/reviews/adata-xpg-sx8200-pro-ssd,5955-4.html

Heyyy Tom's Hardware has actual benchmark data to run with. Based on that data, the ASX8200PNP-2TT-C you're suggesting seems like a very good one. Going with that model. It's a counter-intuitive pick, being one the cheapest 2TB's out there. The 970 EVO Plus is better but nearly double the cost.

Cell technologies (TLC etc) summarized here:
https://www.tomshardware.com/reviews/ssd-buying-guide,5602.html

[Lenclume]

Seagate is constantly so far ahead at producing junk, there is no competition.

I don't know how these guys are still in business.

[Wuerstchenhund]

I'm not sure I would consider WDC "reputed" when it comes to SSDs, and their reputation with spinning rust is somewhat questionable (of all the hard disk manufacturers that are still in existence, WDC has given us by far the most trouble over the last 20 years or so).

Seagate is constantly so far ahead at producing junk, there is no competition.

Be careful with those Backblaze stats (not just because the ones you referred to are for drives that have been in production for years now), because as a cloud provider their use case (i.e. cheap SATA drives for high load mass storage in large scale disk arrays) isn't really representative of other usage scenarios. Backblaze does it not because it's the most reliable setup but simply because it's cheap enough so they can compete with other cloud offerings.

Interestingly, it appears Backblaze no longer uses WDC drives (although in a way they do as HGST is now part of WDC) but still use Seagate drives:



And while an older model of Seagate's 12TB drive has a higher failure rate than the rest it's still not excessive, considering the environment these disks are used (also, Seagate drives tend to be notably cheaper than everyone else, which means HGSTs slightly lower failure rate comes at a premium).

It's also worth looking at the AFRs over the years:



And here both WDC models show an AFR as high as the less reliable Seagate drives. So there's that.

But yes, you are right that Seagate had some models that were failure prone (like that old Barracuda 7200.11, one of the first hard drives with capacities over 1TB, from the Blackblaze list, although that one was really a particularly bad outlier). So did everyone else, including WDC (like many of their Green Series drives which failed quicker than you could order replacements, same for the very expensive Raptor Series).

But quite a few times over the years WDC has also caused us other errors where drives (specific models, not just an individual drive) were not just failing but causing some strange errors, sometimes only after a while. We've seen no such effects with any drive from any other manufacturer.

On top of that, there is now the SMR debacle.

Considering that SSDs are available from a lot more vendors than hard drives there's really no need to risk it with WDC.

[Wuerstchenhund]

Like retention time which is hardly ever specified

It's specified for enterprise-grade SSDs and usually sits somewhere between 1 and 2 years (unpowered). Which means flash based drives make for very poor archiving storage.

I suspect endurance is routinely exaggerated.

I doubt that. Endurance can easily be calculated based on the write endurance of the flash memory the SSD manufacturer buys and the amount of over-provisioning that exists.

Endurance ratings are generally very conservative and drives tend to survive for a lot longer than specified, unless it's one of a group of intel SSDs which block any write activity once the endurance counter has reached zero.

But a high endurance rating doesn't mean a drive won't die because of anything else before it reaches its limit.

[wraper]

And while an older model of Seagate's 12TB drive has a higher failure rate than the rest it's still not excessive, considering the environment these disks are used (also, Seagate drives tend to be notably cheaper than everyone else, which means HGSTs slightly lower failure rate comes at a premium).

And you still can see that Seagate HDD have much larger failure rate among all models. BTW I've seen plenty of Seagate HDD fail and have a few of them which developed bad sectors laying around. Also it's not like Seagate made one bad series and then it became OK. They repeated it several times. Backblaze buys any HDD they can get cheap, they will buy crap as long as cost benefits outweigh losses due to failure rate. IIRC they once made a statement about that.

also, Seagate drives tend to be notably cheaper than everyone else, which means HGSTs slightly lower failure rate comes at a premium

It's not slightly lower. It's about 3-4 times lower on average if not considering those atrocious Seagate HDD from earlier years.

[David Hess]

I suspect endurance is routinely exaggerated.

I doubt that. Endurance can easily be calculated based on the write endurance of the flash memory the SSD manufacturer buys and the amount of over-provisioning that exists.

Endurance ratings are generally very conservative and drives tend to survive for a lot longer than specified, unless it's one of a group of intel SSDs which block any write activity once the endurance counter has reached zero.

The problem is that on the drives I have, the specified endurance does not match up with the drives reported remaining operating life compared to how much data has been written.

It might be due to write amplification in my application but so what?  That still means I cannot rely on the published endurance specification.

[Wuerstchenhund]

And while an older model of Seagate's 12TB drive has a higher failure rate than the rest it's still not excessive, considering the environment these disks are used (also, Seagate drives tend to be notably cheaper than everyone else, which means HGSTs slightly lower failure rate comes at a premium).

And you still can see that Seagate HDD have much larger failure rate among all models.

No, they don't. And not just because Backblaze only lists stats for a very limited number of models. Stats which, for example, for 2017 list an AFR of 0.89% for their 10TB Seagate Barracuda Pro (ST1000NM0086) which is lower than all three of the listed WDC drives, of which two have an AFR over 2%. So much to "among all models".

And this leaves out all the validity issues with Backblaze's data in the first place.

BTW I've seen plenty of Seagate HDD fail and have a few of them which developed bad sectors laying around. Also it's not like Seagate made one bad series and then it became OK. They repeated it several times. Backblaze buys any HDD they can get cheap, they will buy crap as long as cost benefits outweigh losses due to failure rate. IIRC they once made a statement about that.

Yes, and they also put lots of cheap consumer grade drives in large disk arrays where they are exposed to constant vibrations for which they were never designed for.

Also, Backblaze doesn't list other relevant data like production date/manufacturing charge, firmware level and other things which would be relevant as they affect AFR. 

Which is why I already said you have to take their findings with a large grain of salt, because unless you use hard disks in a similar environment then they are meaningless.

BTW, there's a good, albeit older, article discussing the issues with Backblaze's stats:

https://www.tweaktown.com/articles/6028/dispelling-backblaze-s-hdd-reliability-myth-the-real-story-covered/index2.html

also, Seagate drives tend to be notably cheaper than everyone else, which means HGSTs slightly lower failure rate comes at a premium

It's not slightly lower. It's about 3-4 times lower on average if not considering those atrocious Seagate HDD from earlier years.

No, it's not. We use a lot of spinning rust in various use cases distributed all around the Western hemisphere (we have >10k drives in the US alone), and I have seen the stats and the majority of drives are within equal margins (but unlike Backblaze we don't shuck cheap external hard drives, we don't buy refurbs, and we don't use desktop drives in a vibration-heavy rack environment; we also record mfg dates/batch numbers, firmware versions and so on, and compensate for environmental factors). And Seagate is really no different than the other manufacturers aside from WDC (because of the strange problems I mentioned earlier). Of course there always can be the odd outliers but as long as its within a reasonable margin (not exceeding 3.5%) we simply don't give a shit. Hard disks are consumables, and notoriously unreliable ones at that.

Because, at the end of the day, even an AFR of 0.09% doesn't mean that the three drives you just bought won't all die within the first week. AFR is a statistical value across a large number of disks, not a guarantee that the specific exemplar you bought will show the same reliability. AFR can be a useful parameter when you buy large amounts of disks (i.e. in the hundreds or more). It's completely meaningless for home users which just buy two, three, four or five drives (a number way too small to derive any useful reliability data from) for their home server or NAS, and who rarely understand all the things that affect AFR anyways.

So all the faffing about AFR is a waste of time, really.

[Wuerstchenhund]

The problem is that on the drives I have, the specified endurance does not match up with the drives reported remaining operating life compared to how much data has been written.

It might be due to write amplification in my application but so what?  That still means I cannot rely on the published endurance specification.

Have you checked the drive's temperature? Because high temperatures can drastically reduce flash memory endurance, and SSDs can produce a lot of heat (some need active cooling) during use (most drives also start to throttle write speeds when overheating). If your app writes a lot of data then the drive will get hotter, how much depends on the memory that's used, the drive's mechanical design and the airflow it gets in your computer.

And of course, flash memory can always fail prematurely. Failed memory cells reduces the amount of spare memory and thereby your endurance.

[wraper]

The problem is that on the drives I have, the specified endurance does not match up with the drives reported remaining operating life compared to how much data has been written.

It might be due to write amplification in my application but so what?  That still means I cannot rely on the published endurance specification.

Have you checked the drive's temperature? Because high temperatures can drastically reduce flash memory endurance, and SSDs can produce a lot of heat (some need active cooling) during use (most drives also start to throttle write speeds when overheating). If your app writes a lot of data then the drive will get hotter, how much depends on the memory that's used, the drive's mechanical design and the airflow it gets in your computer.

And of course, flash memory can always fail prematurely. Failed memory cells reduces the amount of spare memory and thereby your endurance.

Actually flash memory endurance (not retention) is the best when the flash chips are a little bit hot. Also they do not heat much, it's a controller which gets hot and starts to throttle, nothing to do with temperature of the flash chips.

[wraper]

The problem is that on the drives I have, the specified endurance does not match up with the drives reported remaining operating life compared to how much data has been written.

It might be due to write amplification in my application but so what?  That still means I cannot rely on the published endurance specification.

Dunno what figures you see, but according to reviewer tests, SSD in general survive several times, sometimes even more than 10 times more data written than TBW spec before starting having issues.

[Wuerstchenhund]

Actually flash memory endurance (not retention) is the best when the flash chips are a little bit hot.

Actually, temerature doesn't matter much for endurance unless the temperature profile is notably exceeded (i.e. operating above the spec'd limit), where cells can suffer damage, reducing the amount of spare cells (and thereby the drive's life).

Temperature however matters a lot for retention, and depending what you mean with "a little hot" you are right. Physics tells us that the aount of charge a flash memory cell can hold decreases at very low (i.e. below 10deg C) or very high (i.e. 70deg C) temperatures, so (depending on the memory type), so ideally it's kept somewhere in between. Most TLC and other modern day flash memory that's used in SSDs is only specified for between up to 60 degree or lower and above that will quickly lose its charge capacity.

Also they do not heat much, it's a controller which gets hot and starts to throttle, nothing to do with temperature of the flash chips.

This is not correct. While the controller itself can (and often does) get hot during operation, flash memory does as well.

On fast enterprise NVMe drives like the Micron 9200 Series U.2 drives the flash memory gets so hot that the drive requires active cooling, despite having a metal housing with embedded cooling fins which acts as a heat sink. Without active cooling the drive literally gets too hot to touch during operation.

Enterprise class high performance drives often have a setting where the power consumption (and thereby the amount of heat that's produced) can be limited in certain steps where the controller throttles the drive once a specific power draw is reached.

Consumer-grade drives usually have fixed throttling schemes which can't be configured and which start to throttle the drive as soon as the flash memory reaches a certain temperature. Because for these drives that temperature is usually fixed at a lower value long before it gets critical for the memory, the flash memory will remain comparatively cool.

[wraper]

On fast enterprise NVMe drives like the Micron 9200 Series U.2 drives the flash memory gets so hot that the drive requires active cooling, despite having a metal housing with embedded cooling fins which acts as a heat sink. Without active cooling the drive literally gets too hot to touch during operation.

And where did you get the info that it's Flash that gets hot, not controller? There is nothing exceptional about it's write speeds compared to regular NVMe SSD. IME on NVME SSDs while continuously writing at max speeds Flash ICs become simply warm while controllers easily run at 100^oC

as soon as the flash memory reaches a certain temperature.

Except there is no temperature sensor for Flash normally.

[wraper]

BTW I've seen plenty of Seagate HDD fail and have a few of them which developed bad sectors laying around. Also it's not like Seagate made one bad series and then it became OK. They repeated it several times. Backblaze buys any HDD they can get cheap, they will buy crap as long as cost benefits outweigh losses due to failure rate. IIRC they once made a statement about that.

Yes, and they also put lots of cheap consumer grade drives in large disk arrays where they are exposed to constant vibrations for which they were never designed for.

Those bad drives which I have were not exposed to anything, one HDD per office computer at my work.

[David Hess]

The problem is that on the drives I have, the specified endurance does not match up with the drives reported remaining operating life compared to how much data has been written.

It might be due to write amplification in my application but so what?  That still means I cannot rely on the published endurance specification.

Have you checked the drive's temperature? Because high temperatures can drastically reduce flash memory endurance, and SSDs can produce a lot of heat (some need active cooling) during use (most drives also start to throttle write speeds when overheating). If your app writes a lot of data then the drive will get hotter, how much depends on the memory that's used, the drive's mechanical design and the airflow it gets in your computer.

And of course, flash memory can always fail prematurely. Failed memory cells reduces the amount of spare memory and thereby your endurance.

The drive I first noticed this one was in the open as part of a test system representing a best case scenario.  And this was not a grade z drive; it is a Micron BX500. I have since switched to MX500 drives to see if the DRAM helps.

Do the drives include temperature in their calculation of remaining operating life?

Shouldn't the advertised endurance specification include all factors which affect the endurance?

[Halcyon]

TBW shouldn't be used (on its own) as an indicator of drive quality or endurance. It's largely a number made up by the manufacturers where they guarantee the drive against failure for that about of TB written to the NAND. If a drive falls short of that figure, it falls under their standard warranty and is replaced. It's a bit of a cost vs. benefit analysts on their part where through R&D and testing, they expect x% of drives to fail under warranty in a given period and factor that in to the price of their products. If they set that TBW figure too high, they might find themselves in a situation where more drives are deemed faulty towards the end of their usable life and starts eating into their profit margins, too low and it looks bad for marketing.

Take it with a grain of salt. NAND quality does vary and there are reasons why cheap Chinese drives are cheap.