Author Topic: modern storage is crazy  (Read 6103 times)

0 Members and 1 Guest are viewing this topic.

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13157
  • Country: ch
Re: modern storage is crazy
« Reply #50 on: April 29, 2022, 06:23:55 pm »
Anyway, what I am doing is not *THE* point, but rather if those modern 14TB disks are as reliable as the smaller ones I am using.

Personally I don't think so; as I understand it, modern disks are less reliable, so let's fix the problem with RAID? For example, RAID-6 appears to be mandatory for that Synology 8-bay drive.
Whatever gave you the idea that modern disks are less reliable?!?

For years and years, the only pattern in disk failure has been that some individual models have proven to be abnormally unreliable in the long run. But those failures have not, in the long run, shown there to be any particularly good or bad eras for the whole industry, and rarely even for a brand. (With the arguable exception of the IBM/Hitachi/WD DeskStar models, which have had exceptionally good reliability for the past decade or two, after a single bad model around the year 2000.)
 
The following users thanked this post: james_s

Online jpanhalt

  • Super Contributor
  • ***
  • Posts: 4002
  • Country: us
Re: modern storage is crazy
« Reply #51 on: April 29, 2022, 06:28:01 pm »
Pure dogma and semantics.  What happens when backups are destroyed in a fire or flood?  Same scenario.  There are degrees of safety offered by redundancy.  Backups are just another form of redundancy.  As reliability of storage increases, the need for multiple methods of redundancy decreases.

An analogy can be made to trans-ocean commercial flights.  Used to require more than 2 engines.  Now just 2 is OK.  Engines are more reliable today.
 
The following users thanked this post: eugene

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7508
  • Country: va
Re: modern storage is crazy
« Reply #52 on: April 29, 2022, 08:55:03 pm »
Quote
An analogy can be made to trans-ocean commercial flights.  Used to require more than 2 engines.

That's not an appropriate analogy. And that's assuming you equate engines to storage means - if you actually relate them to drives then it's even less appropriate.

I think the web fora go-to analogy involves a motor car, if you can figure one out... :)
 

Online jpanhalt

  • Super Contributor
  • ***
  • Posts: 4002
  • Country: us
Re: modern storage is crazy
« Reply #53 on: April 29, 2022, 09:30:49 pm »
What about "redundancy" did you miss?
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7508
  • Country: va
Re: modern storage is crazy
« Reply #54 on: April 29, 2022, 11:20:52 pm »
So, what do engines relate to? Are you saying that with more reliable drives we don't need as many backups? Surely if drives are becoming that reliable we wouldn't be needing every-increasing numbers of units in a RAID array and could manage with just RAID1.

But we'd still need backups just as regularly as now.

Quote
What about "redundancy" did you miss?

Well this is why I suggested your analogy was inappropriate. You seem to be saying that because engines are more reliable we only need two whereas before we needed more. Which literally translates, as noted above, to fewer drives in an array. But for data protection it is meaningless - if you fly through a cloud of volcanic ash, two or four engines isn't going make a lot of difference. A bunch of geese on takeoff will still see you in the Hudson. Maybe you meant parachutes were still needed despite better engines?
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: modern storage is crazy
« Reply #55 on: April 30, 2022, 12:22:16 am »
If drives had gotten more reliable the need for backups is decreased. I don't think drive reliability has changed substantially over the last 30 years or so though. Nothing is ever perfect, backups of critical data are always good to have, although there is a broad range of data that is technically replaceable but inconvenient, so one has to weigh the effort of keeping meticulous backups against the effort of reconstructing if something fails.
 

Offline amyk

  • Super Contributor
  • ***
  • Posts: 8526
Re: modern storage is crazy
« Reply #56 on: April 30, 2022, 02:24:59 am »
I suppose the OP is looking for some 4MB memory cards too.  :-DD

Just kidding around; I totally get his frustration. Ten years ago I bought a Synology NAS and stuffed two 2TB drives into it (RAID 1.) I thought 2 TB was ridiculous at the time and I was right; the drive is still only half full. The really sad thing is that the drives are 5.25". If one of them fails I'm not sure I could even find a replacement!
You definitely did not buy 5.25” hard disk drives for your NAS ten years ago, since they stopped making 5.25” hard disks over 22 years ago (and even then they were extraordinarily rare). Not even old stock, since the last one of those was just 19GB.
They made 5.25" disks up to 47GB:



If drive manufacturers were interested in using 5.25" platters again, 47TB would probably fit in that size. The drive above has 14 platters.
 

Offline Ranayna

  • Frequent Contributor
  • **
  • Posts: 986
  • Country: de
Re: modern storage is crazy
« Reply #57 on: April 30, 2022, 07:41:50 am »
If drives had gotten more reliable the need for backups is decreased. [...]
The need for backups is only tangentially related to the reliability of drives.
I personally never had a drive fail or other catastrophic hardware failure, but i still lost data :p

There are just too many ways of losing your data besides drive failures, that backup of important data is mandatory.

If drive manufacturers were interested in using 5.25" platters again, 47TB would probably fit in that size. The drive above has 14 platters.
5 1/4" drives would be slooooooow, compared to 3.5" disks, and spinning rust is already slow compared to SSDs. The seek times would be significantly longer, and also the large platters cannot be driven to the high RPMs of modern disks without starting to wobble.
There is a reason why, back in the day, the highest performance harddisks generally were 2.5". Even many 3.5" harddisks used smaller platters.
 
The following users thanked this post: tooki, DiTBho

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #58 on: April 30, 2022, 11:15:56 am »
I don't want a fast disk, I want a reliable disk!
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Online mariush

  • Super Contributor
  • ***
  • Posts: 5170
  • Country: ro
  • .
Re: modern storage is crazy
« Reply #59 on: April 30, 2022, 11:24:40 am »
I don't want a fast disk, I want a reliable disk!

Would you spend $300 on a 7-10 year warranty "enterprise" drive, $200 on a "nas grade" 5 year warranty drive or $150 on a 2-3 year warranty "desktop" grade drive?

That's the problem...  people chase the lowest prices.
Personally, I go for "nas grade" drives for longer warranty and don't mind paying 20-50$ over the price of a cheap drive. I recently went and bought a 4TB WD Red Plus drive because I didn't want SMR drive, and those Red Plus are guaranteed to be CMR drives (and most 8 TB and higher are CMR in WD's lineup)
I would rather spend $400 for 2 drives instead of a single enterprise drive, because at least if one drive fails, there's the other as spare until the warranty replaces the drive or I buy another.
The "enterprise"  drives may be better or faster, but they still fail, infant mortality, random failures, their MTBF is just a bit better than the other drives MTBF, you're not guaranteed it won't fail.
 
The following users thanked this post: tooki, DiTBho

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13157
  • Country: ch
Re: modern storage is crazy
« Reply #60 on: April 30, 2022, 12:39:52 pm »
Pure dogma and semantics.  What happens when backups are destroyed in a fire or flood?  Same scenario.  There are degrees of safety offered by redundancy.  Backups are just another form of redundancy.  As reliability of storage increases, the need for multiple methods of redundancy decreases.
No, it's not just "dogma and semantics".

RAID protects against precisely one thing, and nothing more: drive failure.

It doesn't protect against any other form of data loss. It doesn't protect against natural disasters, theft, software failure, or human error.

The key difference between backups and redundant storage is that in the latter, the redundancy data is always updated simultaneously with the original. (In the case of mirroring, the mirrored copy is updated simultaneously.) It's redundancy of your normal working data.

Backups, on the other hand, are separate copies made periodically, but not in real-time. In proper backup systems, versioning is also implemented, so that rolling back to a prior version is possible.
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13157
  • Country: ch
Re: modern storage is crazy
« Reply #61 on: April 30, 2022, 12:43:13 pm »
If drives had gotten more reliable the need for backups is decreased. I don't think drive reliability has changed substantially over the last 30 years or so though. Nothing is ever perfect, backups of critical data are always good to have, although there is a broad range of data that is technically replaceable but inconvenient, so one has to weigh the effort of keeping meticulous backups against the effort of reconstructing if something fails.
Nonsense. Drive failure is but one of a multitude of reasons why backups are needed. I've been lucky and have never lost data to drive failure (I replace a drive as soon as any of the critical SMART parameters start to rise, even if the disk has not yet thrown any disk errors, let alone failed). But I have used backups many times to recover things I edited and then wanted an old version of, or from program crashes or bugs.
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13157
  • Country: ch
Re: modern storage is crazy
« Reply #62 on: April 30, 2022, 12:50:05 pm »
I suppose the OP is looking for some 4MB memory cards too.  :-DD

Just kidding around; I totally get his frustration. Ten years ago I bought a Synology NAS and stuffed two 2TB drives into it (RAID 1.) I thought 2 TB was ridiculous at the time and I was right; the drive is still only half full. The really sad thing is that the drives are 5.25". If one of them fails I'm not sure I could even find a replacement!
You definitely did not buy 5.25” hard disk drives for your NAS ten years ago, since they stopped making 5.25” hard disks over 22 years ago (and even then they were extraordinarily rare). Not even old stock, since the last one of those was just 19GB.
They made 5.25" disks up to 47GB:

[video]
Sure, but the last 5.25" drives made were the Quantum Bigfoots, which topped out at 19GB, according to the interwebs.

If drive manufacturers were interested in using 5.25" platters again, 47TB would probably fit in that size. The drive above has 14 platters.
By the 1990s it was clear that there was no interest in doing that, and then not long after started going the other way, towards 2.5" platters for server and enthusiast disks.

Additionally, the economies of scale of 3.5" drives meant that merging a bunch of those into a RAID array is cheaper than the equivalent storage might have cost in a low-volume giant server disk. But the RAID is not only cheaper, but also far faster, and can add redundancy.

I also would expect that the manufacturing yields on large platters are disproportionately higher, for the same reason that physically large ICs have lower yields, because the chances of having a bad region on a big disk are higher, for a given defect rate per surface area.
« Last Edit: April 30, 2022, 12:54:44 pm by tooki »
 

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #63 on: April 30, 2022, 12:51:07 pm »
4TB WD Red Plus drive

I am reading that Seagate put a kind of protective layer on the top of the magnetic platters in their 7200.xx Barracuda lines, but without testing the reliability of the solution, and as result, the protective layer can seriously damage the tiny heads, so one day you experiment the BSY-bug, which is not only a firmware problem because behind it there is that quality of your hard-disk platters sucks.

Built and tested for decent quality rather than built, poorly tested, but promoted to be great, crazily fast and cheap!
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 13157
  • Country: ch
Re: modern storage is crazy
« Reply #64 on: April 30, 2022, 12:52:59 pm »
I don't want a fast disk, I want a reliable disk!
Well, you can buy some high-reliability enterprise disks (some types do have orders of magnitude better MTBF specs than others!), but for most things, you're better off with an array or SSD.

Plus a backup. Well, two backups if you wanna do things right: one at home and one off-site. (I keep my offsite backups in my locker at work and bring them home every 2 or 3 weeks to update them.)
 
The following users thanked this post: DiTBho

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #65 on: April 30, 2022, 12:58:48 pm »
Well, you can buy some high-reliability enterprise disks

SAS instead of sATA?
or FC-disks?
can you suggest something?  :D
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Online mariush

  • Super Contributor
  • ***
  • Posts: 5170
  • Country: ro
  • .
Re: modern storage is crazy
« Reply #66 on: April 30, 2022, 03:09:38 pm »
WD Gold Enterprise drives are specc'ed as this :  datasheet

Load/unload cycles 600,000 
Non-recoverable read errors per bits read <1 in 1015
MTBF (hours) 2,500,000 
AFR (%) 0.35 
Limited warranty (years)  5

Workload  (in fineprint)  219 TB per year

WD Red Plus : datasheet

Reliability/Data Integrity
Load/unload cycles4 600,000
Non-recoverable errors per bits read <1 in 1014
MTBF (hours)5 1,000,000 
Workload Rate (TB/year)  180
Limited warranty (years)  3

AFR not specified ..

So you can see  2.5 M  vs 1M MTBF , 219 TB/year vs 180 TB/year  etc etc ...  In theory the enteprise drive is better. 

In practice, you could buy 10 enterprise drives and 10 nas drives and have more enterprise drives fail on you within a year. 

Backblaze publishes stats for their inventory of drives, and their AFR is a bit higher, at around 0.5-0.7, with Seagate being much worse, close to 1-2% : https://www.backblaze.com/blog/backblaze-drive-stats-for-2021/

But take their results with a big grain of salt, because they don't use the hard drives as a typical person would use them - they have 4U cases where they shove 40+ drives in a case, so they're probably running much warmer than normal, and there's significantly more vibrations compared to what a drive would experience in a normal case, or in a max 8 drive NAS system.

They had 202759 drives in 2021, and 1820 died ... 1.01 AFR average, but Seagate drives peaked at 4.8% AFR ( 77 out of 1611 14 TB drives died)
You'll find HGST drives ( now WD Gold and other higher end models) among the lowest failure drives.



 
The following users thanked this post: tooki, DiTBho

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: modern storage is crazy
« Reply #67 on: April 30, 2022, 06:48:59 pm »
5 1/4" drives would be slooooooow, compared to 3.5" disks, and spinning rust is already slow compared to SSDs. The seek times would be significantly longer, and also the large platters cannot be driven to the high RPMs of modern disks without starting to wobble.
There is a reason why, back in the day, the highest performance harddisks generally were 2.5". Even many 3.5" harddisks used smaller platters.

In a lot of cases that doesn't matter. I use SMR drives for backing up my media server, they're slow, but I don't care, I just kick off the backup and a few days later it's finished. If I could get a single 16+TB 5.25" drive that cost less than a pair of 8TB 3.5" drives I would seriously consider it. Doubt this will happen though.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf