tools to understand when a hard drive is close to death

[DiTBho]

Code: [Select]

is_ok &= bought_it_new_from_respectable_distributor && didnt_turn_up_loose_in_jiffy_bag;


new is_OK_conditions added 

... but is_respectable_distributor() needs more study; look at the conversation with the seller: it's not enough to trust someone's words

[bd139]

That seller's email is hilarious. He either doesn't know or is lying. Either way incompetence is not an excuse 

[DiTBho]

That seller's email is hilarious. He either doesn't know or is lying. Either way incompetence is not an excuse 

And he didn't reply to further emails.
People like him are a plague for the eCommerce.

[bd139]

Just remember that bad reviews travel faster than the speed of light on the internet 

[DiTBho]

Anyway, discussing the development of the disk testing program, there are other interesting fields that may be worth considering

• accumulated start-stop cycles (currently ignored)
• total correction, errors algorithm, corrected invocations (currently ignored)
• total uncorrected errors (currently ignored)
• elements in grown defect list (currently ignored)

In this case, I need trigger values

[bd139]

I would throw something like the following in. That gives a chance for the vendor to actually test the disk before sending it out as it's probably not shipped from the manufacturer.

Start-stop < 20

corrected errors > 10

uncorrected errors > 0

grown defect list > 0

Also to note I had some problems with actual HPE shipped SAS disks a couple of years back. The bearings seized after a few power cycles because they were stuck in a warehouse for half a decade. So storage is a consideration that needs to be made too. This put me right off any mechanical disks. In fact we have no mechanical disks at all now.

[DiTBho]

I will add your acceptance criteria

Start-stop < 20
corrected errors > 10
uncorrected errors > 0
grown defect list > 0

Thanks, this way I can set a solid QA level of acceptance 


Hope to have enough time to also support SAS, sATA, and FC disks (they have different queries).
May be I will release the final program OpenSource. Once polished and completed.

[DiTBho]

stuck in a warehouse for half a decade. So storage is a consideration that needs to be made too

I can consider this only by monitoring the phase-tag errors reported by the kernel.

It only works for SCSI, it's not rock-science but it can somehow summarize some possible problems concerning the aging of components: oxidation of connector, or deterioration of electronics or heads.
They can manifest in a sneaky way, not enough to cause a failure, but enough to cause too many retries, and in this case the kernel can report the problem via the SCSI verbose module.

I will add soon 

[bd139]

Worth pointing out active monitoring here.

Last place I worked were using this: https://github.com/prometheus-community/smartctl_exporter

That then gets scraped by prometheus and alerted on by alertmanager. So when the DB SQL write master disks finally gave out, it'd autonomously wake up a troll who went and failed the node over and swapped the disk out.

But they moved to EC2 / EBS where disk failure is Amazon's problem as it's a logical volume abstraction 

[DiTBho]

Worth pointing out active monitoring here.
Last place I worked were using this: https://github.com/prometheus-community/smartctl_exporter

Yup, that is step3

step1: acquire disks
step2: put disks inside the Rack
step3: monitor disks

it's interesting code, unfortunately written in GoLang, a language that I cannot use because there is no Clang support for { HPPA2 , MIPS }.

I will study the code, anyway. Thank your for posting 

[bd139]

You love to hurt yourself don't you 

SNMP + smartctl hackery it is 

[MrMobodies]

DiTBho where did you get it from? Sorry I didn't see there were three pages already.

I take pictures from the time I receive stuff to the testing. For something like that I'd start a surface scan with something like MHDD. I have a machine dedicated just for that for many years with a Scsi controller that works with it.

https://hddguru.com/software/2005.10.02-MHDD/

No I don't need to test it if there are signs of use but I do it to add weight to my argument of how used it is and I had a quite a few that were very well used so they can't come close to being described as "new".  I expect it to help speed up a returns delivery form without any arguments and to eliminate having to hear the following claptrap excuses such as; well I have used it a little here and there, it's an ex demo etc...

[MrMobodies]

Which SCSI interface/adapter are you running?

Back in the mid '90s, I had SCSI II gear. Big HDD and Tape backup. Started out with a ISA -> SCSI adapter (adaptec?). Whilst it worked fine on 16-bit winders, the win 95 drivers or implementation was cancer. It was hogging the DMA and causing all sorts of mischief until I got the fancy PCI -> SCSI (1540?) adapter. Happy days.

I still have an old RAID system like this intact and stored away. I got them in 2006. It is in a Coolermaster stacker case with a Tyan S2720 serverboard, Dual Xeon 3Ghz and 6gb ram with a Megaraid Enterprise 1600 (128mb battery backed up sdram) SCSI 2 with 6x 32gb Fujitsu 15000rpm drives in raid 5 and many cooling fans in the front to cool the drives. I got the board and controller second hand but I think I got the drives new old stock in 2007 for £30 each. I had some IDE drives in there too and was using it as a file server and run experiments on. Seemed quite reliable over the years I was using it. I did have two Seagate's in there both 36gb 15,000rpm as part of the raid at the beginning but they kept on going offline and causing all sorts of problems. When I tested the Seagate's individually they seem okay, when using them alone they are okay so I find that strange despite being the same specification. I stopped using it in 2017 when I brought a HP DL380 G5. That one 6x 72gb scsi3 drives in raid 5. Two of the drives were failing but I picked a couple up for £10 each.

Just looking up the name on the email and according to his Linkedin Profile to the company he works it looks like he's a salesman:
https://www.linkedin.com/in/damian-oduli%C5%84ski-1b0814135/

Damian Oduliński
CIO aPRESS Paweł Sinkiewicz
Sales Manager
aPRESS Paweł Sinkiewicza
Apr 2014 - Present · 8 yrs 3 mosApr 2014 - Present · 8 yrs 3 mos
Żary, woj. lubuskie, PolskaŻary, woj. lubuskie, Polska

I just found the sellers profile:
https://www.ebay.co.uk/usr/apress24

Not looking good.
Other buyers have complained about being sold used as new:

Selling hard drives as new even when they have more than 9000 hours on them. Mine even has the previous owners data on it! Got eBay involved for deceptive listings and ended up getting a full refund. Seller doesn't reply to messages. Give this company a hard pass! Should be ashamed of themselves.
QUANTUM FIREBALL CR 6.4GB 5.4K ATA 3.5'' CR64A02H 655-0695 (#173772525138)
uyer: o***y (139) £55.00   Past month

liar the HDD not working kłamca
SAMSUNG SpinPoint T166 500GB 7.2K 16MB SATA II 3.5'' HD501LJ (#173793516446)
Buyer: i***o (84) £33.00 Past month

The seller lies in the description -> drives have 0 Power_On_Hours , but also show SMART tests that ware done at 20K, 30K or 60 000 hours. So this are not NEW or OPEN BOX drives, but OLD USED drives with cleared SMART values.
HGST UltraStar 7K4000 4TB 7.2k 64MB SATA III 3.5'' HUS724040ALE641 (#175065142498)
Buyer: n***o (102) US $600.00    Past 6 months

S.M.A.R.T. shows the disks have been used for over 7 years, yet sold as new, seller doesn't reply to messages about the issue. Caveat emptor
Fujitsu MAX3147NC 147GB Ultra320 80-PIN 15K RPM (#172697878385) Buyer: k***k (79) £46.00 Past 6 months

Just found their ebay account is in his name:
https://www.ebay.co.uk/usr/apress24

aPRESS - We sell mainly disks to servers and server components.
We provide the highest quality products for a decent price! - We ship all over the world.
===== FAST SHIPPING COURIER UPS ===== WRITTEN GUARANTEE ===== INVOICE FOR EACH PURCHASE =====

Business details:
Business name:   aPress Paweł Sinkiewicz
First name:   Paweł
Last name:   Sinkiewicz
Address:   Ul. Gospodarcza 9
68-200
Żary
PL
Phone:   48684539221
Email:   ebay@apress24.pl

OOps 12 months:
2,570 Positive
27 Neutral
36 Negative

So they have been going since 2011. Top seller rated. If it is true they have been selling all used/damaged drives as brand new look at the amount of people they scammed so far, 2,570.
Many people may not know what to look out of or understand if it is used and works and I have seen this sort of thing before and even sellers that go to lengths to have negative feedback removed.

If you look at a few of their listings.
I all I see are what looks like STOCK photos and I'd be wary when they do that.
I'd only accept stock photos when I buying from somewhere like Ebuyer or Scan or somewhere I know is reputable and get them direct from the manufacturer.

https://www.ebay.co.uk/sch/apress24/m.html?_nkw=&_armrs=1&_ipg=&_from=

https://www.ebay.co.uk/itm/173483626670
Wait a minute, looking at the decorations, the large stock photos and the affiliates.

I suppose I could also get fooled into thinking I am getting old stock or stuff directly from the manufacturers with these people.

[DiTBho]

32gb Fujitsu 15000rpm drives in raid 5 and many cooling fans in the front to cool the drives. I got the board and controller second hand but I think I got the drives new old stock in 2007 for £30 each

You were lucky. I paid 600 euros for 8 discs (~75 euro/disk) , and wasted another 32 euros to return them to the seller via UPS because I have a deadline to meet; I hope Paypal will refund me, and I have yet to find some good SCA disks.

Moral for the story, although it is a very frustrating situation, the upside is that at least it motivated me to write a test program, which is a very educational experience, and very useful 

[DiTBho]

I shipped disks back to the seller, and he (aPress Paweł Sinkiewicz) has escalated this problem to a claim.
Definitively they are not serious! Paypal now is reviewing the information they provided, which means the claim may take longer than 30 days to be resolved.

[bd139]

PayPal will review in your favour. I have had this a few times with sellers. They are usually quite quick as well I.e within a couple of days.

If they work out they’re not making any money out of him after the refund or his bank doesn’t honour the refund transaction they will freeze him, close his account instantly and go “lol” and he’s out of business.

[DiTBho]

Case closed in your favor
We reviewed the case you filed on 29 May 2022 and have decided in your favor.
We've issued a refund to you on . It may take up to 5 days for this refund to be reflected on your PayPal account or bank. If you paid using a credit or debit card, the money will be refunded to your card. Depending on your card issuer, it can take up to 30 days for the refund to appear on your card statement.

It took me a bottle of good wine to persuade my friend lawyer to write an send a letter of judicial formal notice, but it's closed with 100% money back

[Ed.Kloonk]

It took me a bottle of good wine to persuade my friend lawyer to write an send a letter

I can sympathize. Sometimes you have to be as drunk as them to be able to communicate with them.