Author Topic: Synology NAS Western Digital RED Hard Drive FAIL  (Read 7742 times)

0 Members and 1 Guest are viewing this topic.

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Synology NAS Western Digital RED Hard Drive FAIL
« on: June 03, 2021, 07:33:26 am »
A Western Digital RED hard drive in Dave's Synology NAS has finally failed.
Had to happen eventually...

A big controversery in the comments about CMR vs SMR drives. The one that failed is CMR, but I do have one SMR drive in the NAS.

 
The following users thanked this post: SeanB

Offline rolfe

  • Newbie
  • Posts: 6
  • Country: au
  • "This is me"
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #1 on: June 03, 2021, 09:33:25 am »
I had a disk go in one of my NAS boxes at home. When I went to investigate, there was a loud grinding noise coming from the unit.

This is what it looked like (note the prominent groove that has been etched halfway out from the centre!):



After cleaning off the dust(!) this is what one of the platters looked like:



(I was running with 2 redundant units, so getting a replacement and syncing it up was a relatively stress-free process)

Rolfe
 
The following users thanked this post: EEVblog

Offline rdl

  • Super Contributor
  • ***
  • Posts: 3667
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #2 on: June 03, 2021, 09:44:47 am »
How old was it? Everything stops working eventually. I had a Raptor that was 10 years old before I finally replaced it. That's the longest I've ever kept a drive in service. Usually around 5 years or so I start getting paranoid.

edit: I didn't realize there was an embedded video. Only 3 years is a very early death.
« Last Edit: June 03, 2021, 07:40:27 pm by rdl »
 

Offline I_Saldana

  • Contributor
  • Posts: 28
  • Country: be
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #3 on: June 03, 2021, 11:32:09 am »
Never ever would I advide anyone to go for scammy "extended warranty" options....
except for nas harddrives..
If you see models that temporarily offers 5 year warranty instead of (here it's the usual 2 , mandatory) that's the one I'd go for.

I also stumbled on a few posts concerning just this last year when I had a failed zpool.

This is the page I used to check what I should buy as a replacement:

https://nascompares.com/answer/how-to-tell-a-difference-between-dm-smr-and-non-smr-cmr-drives-hdd-compare/
 

Offline newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #4 on: June 03, 2021, 01:25:34 pm »
A big controversery in the comments about CMR vs SMR drives. The one that failed is CMR, but I do have one SMR drive in the NAS.
SMR are horrible for ZFS and bad for anything else. Might be barely tolerable in a desktop.
In a NAS, I would replace it with something better and maybe repurpose it.

Luckily, most vendors are now being a bit more honest in their specification, not hiding the recording mode in a 4 point text in an appendix behind a "Beware of the Leopard" sticker.
E.g. WD and Seagate
« Last Edit: June 03, 2021, 01:27:53 pm by newbrain »
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Online ejeffrey

  • Super Contributor
  • ***
  • Posts: 3719
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #5 on: June 03, 2021, 06:05:23 pm »
A big controversery in the comments about CMR vs SMR drives. The one that failed is CMR, but I do have one SMR drive in the NAS.
SMR are horrible for ZFS and bad for anything else. Might be barely tolerable in a desktop.
In a NAS, I would replace it with something better and maybe repurpose it.

Definitely terrible for ZFS.  I wouldn't say they are automatically bad for NAS, it depends on the application.  For something that is primarily a backup / archival server it would be fine as long as you don't use ZFS.  For a media server that is almost all read access, with occasional bulk writes it might also work.  For a general purpose file server it's no good, and I wouldn't use it on a desktop either.
 

Offline 6gv5

  • Newbie
  • Posts: 5
  • Country: it
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #6 on: June 03, 2021, 07:04:37 pm »
I'm in the process of swapping all my NAS drives due to aging, so I had to go through lots of information about the SMR vs CMR issue during recent months.
Feel free to correct any mistakes.

As far as I can tell, the short story boils down to:

Some vendors (not only WD) started swapping their CMR drives with SMR ones because the technology allows to fit more data in the same space, thus rendering drives cheaper size-wise. So far it seems however WD is the only vendor to clearly indicate which drive uses which technology.

SMR drives aren't inherently defective, but they cannot sustain continuous high loads, which is the case when they're part of a RAID that is being rebuilt. So far it seems they work fine under normal loads in desktop machines, just don't use them in a NAS or a RAID array or put them under heavy load.

CMR are the ones to go, they're good for everything.

Now that is clear that the problem is real, WD segmented further their line of Red drives so that:

"(plain) WD Red" drives are mostly (depending on size) SMR drives, so they're definitely neither intended for continuous load
nor good for RAID arrays. Use them only on desktop machines.

"WD Red Plus" are all CMR plain Red drives relabled as Plus to be easily identifiable, and they're good for NAS/RAID usage.

"WD Red Pro" are also CMR drives, faster also very good for NAS/RAID usage, but aside being faster they're
also more noisy and consume more power, and of course cost more.
Those are the ones to go if one wants maximum reliability and speed.

Given the little difference in price, there would be no reason to buy a plain WD Red drive in place of a Plus one.

Therefore, the Tl;Dr version would be:
1- Don't buy Plain WD Red drives.
2- For Desktops or home/soho NAS use, WD Red Plus are fine.
3- For higher performance/reliability, choose WD Red Pro.

Some information at:
https://shop.westerndigital.com/products/internal-drives/wd-red-plus-sata-3-5-hdd#WD10EFRX
(check the data sheet!)

Corrections welcome!

 
The following users thanked this post: EEVblog, bitwelder

Offline rdl

  • Super Contributor
  • ***
  • Posts: 3667
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #7 on: June 03, 2021, 07:30:23 pm »
True, but it didn't happen until after they got caught trying to sneak it by.

...
So far it seems however WD is the only vendor to clearly indicate which drive uses which technology.
...
 
The following users thanked this post: newbrain

Offline HighVoltage

  • Super Contributor
  • ***
  • Posts: 5473
  • Country: de
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #8 on: June 03, 2021, 07:49:43 pm »
I have a SYNOLOGY authorized vendor that sets up lots of NAS in my town and I bough a new NAS a few weeks ago.

He explained to me that only CMR drives will last a long time in the Synology NAS
I bought these drives:
WD Red Plus WD80EFBX 8TB SATA 6Gb/s SOHO-NAS CMR
The bad thing is that WD does not publish this CMR info on the drive label.


A friend did not want to spend the extra money and bought a few none CMR drives and they lasted less than a month in his Synology NAS.


There are 3 kinds of people in this world, those who can count and those who can not.
 

Offline edavid

  • Super Contributor
  • ***
  • Posts: 3382
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #9 on: June 03, 2021, 07:59:49 pm »
So far it seems however WD is the only vendor to clearly indicate which drive uses which technology.
Seagate was pretty clear about it from when they first started selling SMR drives.

Quote
SMR drives aren't inherently defective, but they cannot sustain continuous high loads, which is the case when they're part of a RAID that is being rebuilt. So far it seems they work fine under normal loads in desktop machines, just don't use them in a NAS or a RAID array or put them under heavy load.
They can't sustain continuous high random write loads.  They are fine for high read loads and for sequential writes.  For a read-mostly RAID 1 array, they are fine for normal operation, but a full rebuild can take 2-3X longer than CMR.  So if you are populating a new RAID 1 system, for a workload that is not random write intensive, it can be quite reasonable to choose SMR drives.  If you ever need to do a rebuild, you can decide then if you want to splurge on a CMR drive, or just wait a little longer for the rebuild.

Quote
CMR are the ones to go, they're good for everything.
Sure, if the price premium is not too high.  If a CMR drive costs twice as much as an SMR drive, maybe not.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #10 on: June 03, 2021, 08:24:56 pm »
How old was it? Everything stops working eventually. I had a Raptor that was 10 years old before I finally replaced it. That's the longest I've ever kept a drive in service. Usually around 5 years or so I start getting paranoid.

edit: I didn't realize there was an embedded video. Only 3 years is a very early death.

I have drives that are over 30 years old and still "in service", in the sense that they are in vintage machines that I fire up once in a while to play with. The oldest drives in anything that I use semi-regularly are around 15 years old. I haven't had many drives fail, especially not recently. Most of those that have died were not very old at the time, most drives I've replaced were just obsolete, they still worked.

As long as you have a backup I don't really see any reason to be paranoid. A drive can fail at any time, and IIRC there is a bell curve of sorts, where most drives that fail do so when they are either relatively new, or quite old.
 

Online rsjsouza

  • Super Contributor
  • ***
  • Posts: 5986
  • Country: us
  • Eternally curious
    • Vbe - vídeo blog eletrônico
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #11 on: June 03, 2021, 08:51:42 pm »
Same here. My oldest is a 1989 Seagate 80MB SCSI-1 HDD, followed closely by a Conner 60MB for a laptop I restored last year.

The durability curve is quite weird - my NAS has been operating since 2012 mostly for backup without a glitch, while my dad's NAS bought at the same time with the same drives died in 2015. Go figure. 
Vbe - vídeo blog eletrônico http://videos.vbeletronico.com

Oh, the "whys" of the datasheets... The information is there not to be an axiomatic truth, but instead each speck of data must be slowly inhaled while carefully performing a deep search inside oneself to find the true metaphysical sense...
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #12 on: June 03, 2021, 09:07:02 pm »
I suspect there are a handful of different failure modes and numerous external factors that come into play. It is also possible that manufacturing variance is a significant factor, hard drives are probably one of if not the most precisely made mechanical devices the average person will ever own. Frankly it's astonishing that they work at all, nevermind withstand years or even decades of continuous operation. I remember being blown away that an entire gigabyte could fit on a single 3.5" drive back in the day, and now we are orders of magnitude beyond that. Terabytes are an absolutely mind boggling number of individual bits, the human brain is not even really able to properly visualize such a vast number. If I hadn't seen them work I would never have believed it was possible to store and recover a specific bit out of trillions stored on a small metal platter. Yet it does work, quite reliably, at astonishing speed, with a completely self contained mechanism that can be had for a couple hundred bucks.
 
The following users thanked this post: EEVblog, rsjsouza

Offline edavid

  • Super Contributor
  • ***
  • Posts: 3382
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #13 on: June 04, 2021, 12:56:22 am »
A friend did not want to spend the extra money and bought a few none CMR drives and they lasted less than a month in his Synology NAS.

Are there people who believe this kind of tall tale  :-//
 

Offline rdl

  • Super Contributor
  • ***
  • Posts: 3667
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #14 on: June 04, 2021, 01:15:19 am »
Maybe the word "paranoid" was a bit too strong. And to be clear, when I say "in service" I mean daily driver service, getting continuously thrashed by the OS. I have drives that date back to the 1990s that still work. The oldest is the 256MB drive that came in my first computer bought in 1993.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #15 on: June 04, 2021, 02:01:22 am »
CMR are the ones to go, they're good for everything.

Now that is clear that the problem is real, WD segmented further their line of Red drives so that:

"(plain) WD Red" drives are mostly (depending on size) SMR drives, so they're definitely neither intended for continuous load
nor good for RAID arrays. Use them only on desktop machines.

"WD Red Plus" are all CMR plain Red drives relabled as Plus to be easily identifiable, and they're good for NAS/RAID usage.

"WD Red Pro" are also CMR drives, faster also very good for NAS/RAID usage, but aside being faster they're
also more noisy and consume more power, and of course cost more.
Those are the ones to go if one wants maximum reliability and speed.

Correct.
I've bought a RED Plus as a replacement.
Thinking about another Red Plus to replace the working SMR in the NAS. But tempting just to leave it until it fails, as I don't notice any performance problems.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #16 on: June 04, 2021, 03:37:00 am »
FYI, the EFAX (SMR) drive runs 4 degC cooler than the EFRX CMR drives.
 
The following users thanked this post: Ed.Kloonk

Offline I_Saldana

  • Contributor
  • Posts: 28
  • Country: be
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #17 on: June 04, 2021, 03:49:21 am »
Thinking about another Red Plus to replace the working SMR in the NAS. But tempting just to leave it until it fails, as I don't notice any performance problems.
Also, depending on the amount of total drives consider a spare for the cupboard. It's not good for warranty but good for your nerves and for the knowledge you can have at least one more fail with the hardware you use still existing.
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14481
  • Country: fr
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #18 on: June 04, 2021, 04:45:58 pm »
I have 3 WD Red HDDs in my NAS. So far so good. But it had WD Green HDDs before, and I had to replace them. WD Green drives were the most horrible drives for NAS use. Although their power consumption was nicely low, their main problem is that they would park their head all the time - and this was not configurable. That would end up ruining the head drive mechanism. This was particularly bad for a home NAS, for which the typical use is NOT constant access to the drives, so that would trigger the head parking much too frequently.

The RED drives do not have this problem, but I know they are not the most reliable out there. I always keep an eye on the drives' SMART data and the state of my RAID.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #19 on: June 04, 2021, 05:11:31 pm »
WD Green are meant for desktop usage. ;)
 

Offline newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #20 on: June 04, 2021, 05:13:38 pm »
WD Green [...] would park their head all the time - and this was not configurable.
A WD utility (WDIDLE or WDIDLE3) could alter some FW parameter to avoid the unwanted head parking.
I have had two WD Green HDs in my NAS until one recently failed*, and both were not parking their heads after using the utility - the difficult part was getting MS-DOS up and having it recognize the drives!

* EtA: not really, just a single checksum error, corrected after a resilver. But I would not trust it any longer.
« Last Edit: June 04, 2021, 05:15:10 pm by newbrain »
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Online ejeffrey

  • Super Contributor
  • ***
  • Posts: 3719
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #21 on: June 04, 2021, 07:55:16 pm »
"WD Red Plus" are all CMR plain Red drives relabled as Plus to be easily identifiable, and they're good for NAS/RAID usage.

"WD Red Pro" are also CMR drives, faster also very good for NAS/RAID usage, but aside being faster they're
also more noisy and consume more power, and of course cost more.
Those are the ones to go if one wants maximum reliability and speed.

WD red plus vs. pro have very similar performance.  The pro does have a slightly higher peak transfer rate at comparable capacity point but that is relatively minor.  The main difference is the Pro has vibration sensors for reliable use in high drive count applications.  It also has longer warranty, so it isn't a bad choice if you want reliable NAS storage but don't expect dramatically performance than the Plus.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #22 on: June 06, 2021, 08:39:41 am »
"WD Red Plus" are all CMR plain Red drives relabled as Plus to be easily identifiable, and they're good for NAS/RAID usage.

"WD Red Pro" are also CMR drives, faster also very good for NAS/RAID usage, but aside being faster they're
also more noisy and consume more power, and of course cost more.
Those are the ones to go if one wants maximum reliability and speed.

WD red plus vs. pro have very similar performance.  The pro does have a slightly higher peak transfer rate at comparable capacity point but that is relatively minor.  The main difference is the Pro has vibration sensors for reliable use in high drive count applications.  It also has longer warranty, so it isn't a bad choice if you want reliable NAS storage but don't expect dramatically performance than the Plus.

I have no need for the high speed. My NAS is only 1Gb ethernet anyway, and the limit is not the drive speed.
 

Offline HighVoltage

  • Super Contributor
  • ***
  • Posts: 5473
  • Country: de
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #23 on: June 06, 2021, 01:05:15 pm »

I have no need for the high speed. My NAS is only 1Gb ethernet anyway, and the limit is not the drive speed.
You might know this already but you can use both of the 1 Gb connections on the Synology NAS and merge them in to one connection to get higher speeds.

 
There are 3 kinds of people in this world, those who can count and those who can not.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #24 on: June 06, 2021, 02:42:31 pm »
... bundle them into a LAG if your switch supports LACP.
 

Offline alexnoot

  • Contributor
  • Posts: 27
  • Country: aq
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #25 on: June 06, 2021, 03:57:57 pm »

I have no need for the high speed. My NAS is only 1Gb ethernet anyway, and the limit is not the drive speed.
You might know this already but you can use both of the 1 Gb connections on the Synology NAS and merge them in to one connection to get higher speeds.

 

I'm sure I heard in the video that he does exactly that?
 
The following users thanked this post: RoGeorge, bdunham7

Offline newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #26 on: June 06, 2021, 04:18:11 pm »
... bundle them into a LAG if your switch supports LACP.
The switch looks like an unmanaged one so no LAG...but SMB mutichannel will automatically use all the links it can find with no problems.
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #27 on: June 07, 2021, 12:30:59 am »
My hard drive hasn't shipped yet, or even progressed through processing yet. I'm startng to think they may not actually have it as claimed...
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #28 on: June 07, 2021, 12:38:35 am »
I have no need for the high speed. My NAS is only 1Gb ethernet anyway, and the limit is not the drive speed.
You might know this already but you can use both of the 1 Gb connections on the Synology NAS and merge them in to one connection to get higher speeds.
I'm sure I heard in the video that he does exactly that?

Yes, I am binding two ethernet ports.
I get 110MB/s read and write.
Actually I just tested it and got the same speed after disconnecting the 2nd ethernet, so  :-//
EDIT: Oh, I just realised that my PC only has the one 1Gb connection, so yeah  :palm:
« Last Edit: June 07, 2021, 12:42:35 am by EEVblog »
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #29 on: June 07, 2021, 12:43:31 am »
My hard drive hasn't shipped yet, or even progressed through processing yet. I'm startng to think they may not actually have it as claimed...

Thank your local Chia idiots for shafting the storage industry for the second time in the last decade.

Can't buy CPUs. Can't buy GPUs. Can't even buy HDDs. Fuck the rest of us, 0.001% of the population has mining to do!
« Last Edit: June 07, 2021, 12:47:06 am by Monkeh »
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #30 on: June 07, 2021, 01:44:46 am »
My hard drive hasn't shipped yet, or even progressed through processing yet. I'm startng to think they may not actually have it as claimed...
Thank your local Chia idiots for shafting the storage industry for the second time in the last decade.
Can't buy CPUs. Can't buy GPUs. Can't even buy HDDs. Fuck the rest of us, 0.001% of the population has mining to do!

I have not looked into that, but have heard it's a thing.
Company said they had stock in the warehouse though...
Have also ordered another RED Plus from another company to replace the SMR drive, that one hasn't shipped yet either... (EDIT: It was just packed)
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16620
  • Country: us
  • DavidH
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #31 on: June 07, 2021, 02:37:46 am »
I have some drives in operation now for 10 years like my original set of 4 x 750G Western Digital Blacks, down to 3, which were the last Black models to support TLER.  After WD updated the firmware to block TLER, I stopped buying from them.  I stopped buying from Seagate after the 7200.10 fiasco.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #32 on: June 07, 2021, 05:15:38 am »
After WD updated the firmware to block TLER, I stopped buying from them.  I stopped buying from Seagate after the 7200.10 fiasco.

What's TLER and 7200.10 fiasco?
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #33 on: June 07, 2021, 07:13:29 am »
After WD updated the firmware to block TLER, I stopped buying from them.  I stopped buying from Seagate after the 7200.10 fiasco.

What's TLER and 7200.10 fiasco?

https://www.abmx.com/blog/what-is-tler#:~:text=This%20feature%20prevents%20hard%20drives,default%2C%20to%20fix%20any%20issues.

 

Offline magic

  • Super Contributor
  • ***
  • Posts: 6779
  • Country: pl
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #34 on: June 07, 2021, 01:11:55 pm »
I think it was the 7200.11 series - the disks sometimes bricked themselves for no reason due to FW bugs.

A WD utility (WDIDLE or WDIDLE3) could alter some FW parameter to avoid the unwanted head parking.
I have had two WD Green HDs in my NAS until one recently failed*, and both were not parking their heads after using the utility - the difficult part was getting MS-DOS up and having it recognize the drives!
If you really can't use DOS there is experimental, reverse engineered support for it in hdparm on Linux.
It worked once for me, though it may be a good idea to look for success stories before trying it on a new model.

Also from personal experience, a pair of Seagate's next generation 7200.12 worked fine for years.
« Last Edit: June 07, 2021, 01:13:29 pm by magic »
 
The following users thanked this post: newbrain

Offline BradC

  • Super Contributor
  • ***
  • Posts: 2106
  • Country: au
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #35 on: June 07, 2021, 01:22:29 pm »
If you really can't use DOS there is experimental, reverse engineered support for it in hdparm on Linux.
It worked once for me, though it may be a good idea to look for success stories before trying it on a new model.

Does the job on any WD Red I've bought in recent history, so still a good'un.

I have a set of 2TB greens from the last run before they disabled TLER, and those have at least 6 years on the clock. The only drives I've had issues with were a set of Maxtor (Seagate 7200.11) 1TB. Other than that, keep them spinning, warm and heads loaded and they go for years.

 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #36 on: June 07, 2021, 02:00:05 pm »
I have some drives in operation now for 10 years like my original set of 4 x 750G Western Digital Blacks, down to 3, which were the last Black models to support TLER.  After WD updated the firmware to block TLER, I stopped buying from them.  I stopped buying from Seagate after the 7200.10 fiasco.

Then you don't have many choices left. >:D WD Black have some features in common with the enterprise/server product lines but aren't designed for NAS/SAN. They are meant for power users and workstations. Seagate possibly screwed up more models than the competitors. However, they also produced a lot of good drives. It's a mixed bag.
 

Offline edavid

  • Super Contributor
  • ***
  • Posts: 3382
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #37 on: June 07, 2021, 03:05:03 pm »
Is there any current NAS device that has such a short, non-configurable timeout that you would need TLER?

(And from my experience, if a drive is in such bad condition that you would ever see TLER in action, it needs to be replaced immediately.)
 

Online ejeffrey

  • Super Contributor
  • ***
  • Posts: 3719
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #38 on: June 07, 2021, 03:59:12 pm »
My hard drive hasn't shipped yet, or even progressed through processing yet. I'm startng to think they may not actually have it as claimed...

Thank your local Chia idiots for shafting the storage industry for the second time in the last decade.

Can't buy CPUs. Can't buy GPUs. Can't even buy HDDs. Fuck the rest of us, 0.001% of the population has mining to do!

(Stolen from reddit) just wait until the next big cryptocurrency comes out based on "proof of case fans"
 
The following users thanked this post: james_s

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #39 on: June 07, 2021, 05:38:54 pm »
My hard drive hasn't shipped yet, or even progressed through processing yet. I'm startng to think they may not actually have it as claimed...

Thank your local Chia idiots for shafting the storage industry for the second time in the last decade.

Can't buy CPUs. Can't buy GPUs. Can't even buy HDDs. Fuck the rest of us, 0.001% of the population has mining to do!

(Stolen from reddit) just wait until the next big cryptocurrency comes out based on "proof of case fans"

This is how I end up bald.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16620
  • Country: us
  • DavidH
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #40 on: June 07, 2021, 07:02:50 pm »
After WD updated the firmware to block TLER, I stopped buying from them.  I stopped buying from Seagate after the 7200.10 fiasco.

What's TLER and 7200.10 fiasco?

TLER is Western Digital's version of error recovery control, Time Limited Error Recovery, which is desirable for RAID.  The Black series of drives supported it until the 750G generation after which it was deliberately removed for purposes of market segmentation.

The Seagate 7200.10 series of drives (or was it the 7200.11 series?) had horrid reliability but Seagate played games to delay returns past their warranty date including releasing faulty firmware updates.

Then you don't have many choices left. >:D WD Black have some features in common with the enterprise/server product lines but aren't designed for NAS/SAN. They are meant for power users and workstations. Seagate possibly screwed up more models than the competitors. However, they also produced a lot of good drives. It's a mixed bag.

The WD Black drives originally supported TLER.

It was not the horribly unreliable and flawed drives that Seagate produced that led me to avoid them in the future.  It was their underhanded tactics to prevent returns and cover up what was going on that convinced me that they could not be trusted.

Since those two events, I have had good results with HGST in various forms.  I am less happy with Samsung but they are no longer a factor.

And of course more recently we have had Western Digital and Seagate (and Toshiba?) concealing which drives use SMR.  That was no oversight.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #41 on: June 07, 2021, 07:26:32 pm »
HGST was bought by WD in 2012 and the brand was phased out in 2018.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16620
  • Country: us
  • DavidH
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #42 on: June 07, 2021, 08:01:44 pm »
HGST was bought by WD in 2012 and the brand was phased out in 2018.

The story is more complex than that with Toshiba receiving the HGST assets for the production of 3.5-inch hard drives.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #43 on: June 07, 2021, 11:09:40 pm »
And sure enough the in-stock claim at Mwave was bullshit, 5 days leter I get the yeah-nah email.
Luckily I ordered another one from another suppliser to replace the SMR and that one has shipped apparently.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16620
  • Country: us
  • DavidH
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #44 on: June 08, 2021, 01:51:31 am »
And sure enough the in-stock claim at Mwave was bullshit, 5 days leter I get the yeah-nah email.
Luckily I ordered another one from another suppliser to replace the SMR and that one has shipped apparently.

I have encountered a bunch of vendors recently though Amazon and Ebay that list items in stock and available when they drop ship and have no idea what is available.  I am not sure what the scam is but am sure there is one; maybe they are farming reputation?  It made for great difficulty in buying a case for my new workstation; there were 10s of deceptive listings to go through and I finally gave up and had to order a different one through a reputable source.
 

Offline BradC

  • Super Contributor
  • ***
  • Posts: 2106
  • Country: au
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #45 on: June 08, 2021, 03:51:14 am »
Is there any current NAS device that has such a short, non-configurable timeout that you would need TLER?

Yeah, most of them. A drive without configured ERC can disappear into the weeds for minutes. What inevitably happens is a read is performed and the drive disappears up its own backside for a while. In the mean time, the NAS tries a reset or a write which fails and it gets kicked from the array before it has a chance to respond.

(And from my experience, if a drive is in such bad condition that you would ever see TLER in action, it needs to be replaced immediately.)

Any read error (recoverable or not) can trigger a delay long enough to cause an issue without TLER. That could have a number of causes, only some of which would indicate any form of pending failure requiring replacement. All TLER does is say "try for x seconds and if you can't get it then give up and I'll find it elsewhere". Incredibly useful behaviour in a redundant array.

Hard drives are electro-mechanical devices. Transient errors happen not at all infrequently. If they didn't the drive manufacturers would not build the firmware to spend a couple of minutes trying to read a difficult sector.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #46 on: June 08, 2021, 04:47:01 am »
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #47 on: June 08, 2021, 05:31:48 am »
The little bit of slop on your driver there is to allow you to get it in and out of the screw. I assure you the ones used in the factory are of much higher quality.

None of the sound you recorded was platter contact, just the servo going nuts. I'm guessing head failure. If it had actually crashed the heads would have been torn to shreds. It might have jammed them up on the spacers, though.
« Last Edit: June 08, 2021, 05:35:42 am by Monkeh »
 
The following users thanked this post: Ed.Kloonk

Offline Ed.Kloonk

  • Super Contributor
  • ***
  • Posts: 4000
  • Country: au
  • Cat video aficionado
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #48 on: June 08, 2021, 07:01:34 am »
The little bit of slop on your driver there is to allow you to get it in and out of the screw. I assure you the ones used in the factory are of much higher quality.

None of the sound you recorded was platter contact, just the servo going nuts. I'm guessing head failure. If it had actually crashed the heads would have been torn to shreds. It might have jammed them up on the spacers, though.

I was wondering about this also. My limited experience of failmodes of WD reds is watching the SMART info from a couple of raids I have and yeah, just listening like Scotty of the Enterprise. When drives went from LBA to whatever voodoo they do now, I remember being told to watch out for drives writing just as a power outage occurs. Apparently, that is drive doom. The head touches the platter for a nano second. No obvious visual evidence. But the damage is done. And you can't even trust the drive reporting anymore since WD knowingly pissed on the WD red brand with the whole SMR bullshit.
iratus parum formica
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #49 on: June 08, 2021, 07:34:01 am »
None of the sound you recorded was platter contact, just the servo going nuts.

Yes, it seems to have recovered somewhat in terms of noise, but it's still way louder than a normal drive. The noise was horrendous though at the time. Maybe it was end stop slamming or something.
 

Offline Ed.Kloonk

  • Super Contributor
  • ***
  • Posts: 4000
  • Country: au
  • Cat video aficionado
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #50 on: June 08, 2021, 07:41:34 am »
None of the sound you recorded was platter contact, just the servo going nuts.

Yes, it seems to have recovered somewhat in terms of noise, but it's still way louder than a normal drive. The noise was horrendous though at the time. Maybe it was end stop slamming or something.

Orientation. Drive manufactures swear blind it doesn't matter. And they might be right, when the drive is healthy.

iratus parum formica
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #51 on: June 08, 2021, 08:01:16 am »
None of the sound you recorded was platter contact, just the servo going nuts.

Yes, it seems to have recovered somewhat in terms of noise, but it's still way louder than a normal drive. The noise was horrendous though at the time. Maybe it was end stop slamming or something.

Orientation. Drive manufactures swear blind it doesn't matter. And they might be right, when the drive is healthy.

That thought did cross my mind.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #52 on: June 08, 2021, 01:17:02 pm »
The little bit of slop on your driver there is to allow you to get it in and out of the screw. I assure you the ones used in the factory are of much higher quality.

None of the sound you recorded was platter contact, just the servo going nuts. I'm guessing head failure. If it had actually crashed the heads would have been torn to shreds. It might have jammed them up on the spacers, though.

I was wondering about this also. My limited experience of failmodes of WD reds is watching the SMART info from a couple of raids I have and yeah, just listening like Scotty of the Enterprise. When drives went from LBA to whatever voodoo they do now, I remember being told to watch out for drives writing just as a power outage occurs. Apparently, that is drive doom. The head touches the platter for a nano second. No obvious visual evidence. But the damage is done. And you can't even trust the drive reporting anymore since WD knowingly pissed on the WD red brand with the whole SMR bullshit.

Makes no difference whether it's reading ot writing, or doing nothing at all. If the power fails in operation the heads should park by themselves long before the platters slow enough cause contact.

SMART data has never been trustworthy. All manufacturers have bugs or intentional omissions, and none publically document the meaning of various values.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 7765
  • Country: de
  • A qualified hobbyist ;)
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #53 on: June 08, 2021, 06:27:20 pm »
IIRC, Google published statistics about HDD failures a few years ago and in about 50% of all cases SMART was able to predict that a HDD will fail.
 

Offline BradC

  • Super Contributor
  • ***
  • Posts: 2106
  • Country: au
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #54 on: June 09, 2021, 12:05:01 am »
IIRC, Google published statistics about HDD failures a few years ago and in about 50% of all cases SMART was able to predict that a HDD will fail.

True, but that wasn’t SMART saying “this drive is about to fail”, it was looking at the stats for an array of identical drives and going “this one is anomalous”. So the failure was predicted by generating statistics across a large population. 50% is still in “flip a coin” territory though. It can help with a proactive “should probably replace this one before it fails” routine, but that’s about it.

In my experience by the time SMART itself reports “failure is imminent” the drive is already in severe distress.
 

Offline amyk

  • Super Contributor
  • ***
  • Posts: 8275
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #55 on: June 09, 2021, 02:00:22 am »
The best predictor of failure is the number of reallocated sectors and "pending sectors". If they go above zero it's time to replace the drive.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #56 on: June 09, 2021, 05:16:40 am »
SMART seems like a good idea, but in practice I don't think it has ever told me anything useful. The few drives I've had fail just completely failed with no warning.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #57 on: June 09, 2021, 09:52:29 am »
Both drives turned up today, just started the sync process, was 1% after 10 minutes.
Using the normal non-priority resync mode. There is an option for faster, but no point stressing the drives I guess.
It uses very little resources on the NAS, 24% CPU and 13% RAM
 

Online rsjsouza

  • Super Contributor
  • ***
  • Posts: 5986
  • Country: us
  • Eternally curious
    • Vbe - vídeo blog eletrônico
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #58 on: June 09, 2021, 10:15:51 am »
SMART seems like a good idea, but in practice I don't think it has ever told me anything useful. The few drives I've had fail just completely failed with no warning.
Same here. I suspect that slow developing issues might be propely flagged, but catastrophic failures are completely off game.
Vbe - vídeo blog eletrônico http://videos.vbeletronico.com

Oh, the "whys" of the datasheets... The information is there not to be an axiomatic truth, but instead each speck of data must be slowly inhaled while carefully performing a deep search inside oneself to find the true metaphysical sense...
 

Offline magic

  • Super Contributor
  • ***
  • Posts: 6779
  • Country: pl
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #59 on: June 09, 2021, 02:50:20 pm »
The best predictor of failure is the number of reallocated sectors and "pending sectors". If they go above zero it's time to replace the drive.
That's not a predictor but an indicator of failure :P
And it may be one-off bit rot. I have a disk which still works after reporting one "pending sector" many years ago. Sector overwritten with new data, problem gone.
 

Online Monkeh

  • Super Contributor
  • ***
  • Posts: 7992
  • Country: gb
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #60 on: June 09, 2021, 02:55:51 pm »
A handful of failed and reallocated sectors is normal and harmless. The drives have spare sectors for a reason.

A rapidly growing count is another matter entirely.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #61 on: June 10, 2021, 02:14:58 am »
Rumors of the death of all the drives in my NAS during resync was greatly exaggerated.
NAS back up and running with a replacement CMR Red Plus drive.

BTW, max drive temp during re-sync was only 2degC above normal idle temp.
And the EFAX SMR drive is only at 15000 hours, so I might just leave it in place.
 

Offline BradC

  • Super Contributor
  • ***
  • Posts: 2106
  • Country: au
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #62 on: June 10, 2021, 03:01:23 am »
And the EFAX SMR drive is only at 15000 hours, so I might just leave it in place.

As already stated, the EFAX drives suck at large random write loads. Rebuilding another drive is a large sequential read load, and they are fine for that. Due to the increase in platter density, they may even be faster than a CMR drive. It's when they need to write randomly they have to "cache" the writes, then when they get some time they re-write the affected zone from the write start point until the end. A bit like an SSD needs to do a read/erase/write on an entire block to change a single "sector" (which is what clever wear leveling works around). SMR needs to re-write everything in the "zone" (think flash erase block size) after the desired write. If the drive knows it's a complete sequential write then it can start at the start and power on through. If it's not sure though it's going to try and write to the "cache" first and then do the proper re-organisation in its idle time.

The issue with SMR drives is in re-build due to the sustained write load. If the NAS vendor has worked with the drive manufacturer to ensure they write in a manner the drive can cope with then it's all ok, provided you aren't doing huge writes to the array while it's rebuilding. They are still considerably slower than CMR drives when you fill up their "cache". Just like TLC/QLC drives are when you fill up the SLC "cache".

 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #63 on: June 10, 2021, 03:10:06 am »
And the EFAX SMR drive is only at 15000 hours, so I might just leave it in place.

As already stated, the EFAX drives suck at large random write loads. Rebuilding another drive is a large sequential read load, and they are fine for that. Due to the increase in platter density, they may even be faster than a CMR drive. It's when they need to write randomly they have to "cache" the writes, then when they get some time they re-write the affected zone from the write start point until the end. A bit like an SSD needs to do a read/erase/write on an entire block to change a single "sector" (which is what clever wear leveling works around). SMR needs to re-write everything in the "zone" (think flash erase block size) after the desired write. If the drive knows it's a complete sequential write then it can start at the start and power on through. If it's not sure though it's going to try and write to the "cache" first and then do the proper re-organisation in its idle time.

The issue with SMR drives is in re-build due to the sustained write load. If the NAS vendor has worked with the drive manufacturer to ensure they write in a manner the drive can cope with then it's all ok, provided you aren't doing huge writes to the array while it's rebuilding. They are still considerably slower than CMR drives when you fill up their "cache". Just like TLC/QLC drives are when you fill up the SLC "cache".

Sure, but why replace it after 15,000 hours if all the SMART tests are fine, zero bad sectors, and I have zero performance issues with it?
If anything, based on the fact that one of the 3 x 26,000 hour CMR drives failed, I should be replacing those instead of the SMR one.
 

Offline BradC

  • Super Contributor
  • ***
  • Posts: 2106
  • Country: au
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #64 on: June 10, 2021, 04:24:00 am »
Sure, but why replace it after 15,000 hours if all the SMART tests are fine, zero bad sectors, and I have zero performance issues with it?
If anything, based on the fact that one of the 3 x 26,000 hour CMR drives failed, I should be replacing those instead of the SMR one.

I wasn't implying you replace it. I was just giving a bit of background on the difference between the two. Far out, it's barely run in. I wouldn't replace the CMR drives either. I have a small array of those 6TB spinners with 36k hours on them. Like most drives, keep them spinning with heads loaded and they'll keep going.

  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36033
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36023
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36023
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36023
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36023
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36031
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36023
  9 Power_On_Hours          0x0032   051   051   000    Old_age   Always       -       36022

I have another "slightly older" set of WD 2TB drives supplemented with some "scraps" I had lying around :
  9 Power_On_Hours          0x0032   071   071   000    Old_age   Always       -       21560
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59698
  9 Power_On_Hours          0x0032   028   028   000    Old_age   Always       -       52862
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59698
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59701
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59729
  9 Power_On_Hours          0x0032   040   040   000    Old_age   Always       -       44217
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59698
  9 Power_On_Hours          0x0032   019   019   000    Old_age   Always       -       59697

As long as they keep going, just use them until they die. I'd avoid buying any new SMR drives though.
 

Offline EEVblogTopic starter

  • Administrator
  • *****
  • Posts: 37740
  • Country: au
    • EEVblog
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #65 on: June 10, 2021, 01:00:52 pm »
As long as they keep going, just use them until they die. I'd avoid buying any new SMR drives though.

That's the plan. Never knew an SMR went into there to begin with, now I know wouldn't buy another one.
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14481
  • Country: fr
Re: Synology NAS Western Digital RED Hard Drive FAIL
« Reply #66 on: June 10, 2021, 05:20:27 pm »
SMART seems like a good idea, but in practice I don't think it has ever told me anything useful. The few drives I've had fail just completely failed with no warning.

I guess the answer is: it depends.
The only failures I've had in about 20 years were those WD Green drives, and in this case, the SMART data showed a definite problem. The number of errors kept growing, and they began to be pretty slow, but they were still readable, so swapping them for new drives in my RAID setting was no problem, without having to use a backup.
The other HDD failure I had was over 20 years ago with a Seagate drive, and at the time, AFAIR, it didn't implement SMART.

 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf