Author Topic: modern storage is crazy  (Read 6100 times)

0 Members and 1 Guest are viewing this topic.

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
modern storage is crazy
« on: April 26, 2022, 10:32:07 am »
__O__M__G__ things like hard drives are going crazy!

This morning I browsed through five different online stores, phoned to six physical ones, and none of them can sell me anything under 2Tbyte!!!

"sorry, the minimal size we have is 2Tbyte"

Which is not exactly correct, Seagate still has some BarraCuda 500GByte HDD in their catalog, but no-store has it for sale, and if you contact a Seagate dealer ... there is a minimal order of 100 units.

So I have just contacted my usual CDD ("computer-drug dealer"), and ordered qty=8 sATA 250GB disks!

(yes,yes,yes, I know ... you can find some on Amazon, but ... I'm paranoid about HDDs ... on Amazon they are usually declared "brand new" while they are second hand, refurbished, cleaned and polished .... sometimes not even very well and find other people's files on the HDD

anyway ...)

Jesus, I have a couple of twin SCSI-to-sATA box, the firmware can only address 1Tbyte of total space, and there are 4 HDD slots, I cannot mount four HDDs of 2TByte each! Neither I can mount 500GByte each (unless I am ready to downgrade the capacity to the half)

4x250GByte = 1TByte -> perfect!
4x500GByte = 2TByte -> on each disk, 50% space wasted

1Tbyte of total space looked so huuuuuuuugggggggeeeeeeee when I bought the storage-box (in 2009), nowadays it's nothing because people is looking for things like the Synology-DiskStation-DS1821 with eight HDD bays, and then each disk is about 14Tbyte

14Tbyte on a single physical HDD, I didn't even think it existed, and that DiskStation has 8 bays of them  :o :o :o

and 8 x 14Tbyte = ....?!?!?#~'~...... WTF <0---+++
it's too big, my mind is 32bit, it overflows

I can't count, can you?  :o :o :o
I can't use such a storage-thing, too big, can you?


Seriously, I cannot even imagine how you can format and use so big storage space.
For Youtube video? Or for what?

Are those huge DISKs reliable? Or too dense (too many Gbyte per mm^2)? Hence prone to failure?

mumble  :-//
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline m98

  • Frequent Contributor
  • **
  • Posts: 634
  • Country: de
Re: modern storage is crazy
« Reply #1 on: April 26, 2022, 10:51:53 am »
I don't see what you're trying to achieve? Why are you buying deliberately small disks for your ancient storage box, instead of just getting a new one? If you want predictable reliability, use RAID 1, 5 or 10.
 
The following users thanked this post: tooki

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #2 on: April 26, 2022, 11:14:35 am »
I don't see what you're trying to achieve? Why are you buying deliberately small disks for your ancient storage box, instead of just getting a new one? If you want predictable reliability, use RAID 1, 5 or 10.

I need to support some old machines, they don't have sATA, they only have SCSI and you can't install any modern PCI / PCIe cards, there are no drivers and it won't work, so a good solution is a sATA-to-SCSI box, which works well, it exposes each sATA disk as if it was a SCSI disk, but the LBA addressing is limited to a quarter of TeraByte on each disk.

Anyway, what I am doing is not *THE* point, but rather if those modern 14TB disks are as reliable as the smaller ones I am using.

Personally I don't think so; as I understand it, modern disks are less reliable, so let's fix the problem with RAID? For example, RAID-6 appears to be mandatory for that Synology 8-bay drive.

Note, I have to replicate the array for a second twin setup, that 1 Tbyte array was created in 2009 and is currently only 40% full, I wonder what people is expecting to store in 112TByte of space.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 
The following users thanked this post: SeanB

Online jpanhalt

  • Super Contributor
  • ***
  • Posts: 4002
  • Country: us
Re: modern storage is crazy
« Reply #3 on: April 26, 2022, 11:37:24 am »
I share the same concern and still use Win7.  My solution is to keep a few 500 gb SSD as spares.  They are still available from Amazon at about $60 locally (Cleveland, Ohio).  I use RAID1.  Just ordered 2 more.
 
The following users thanked this post: DiTBho

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #4 on: April 26, 2022, 11:52:10 am »
They are still available from Amazon

I know ... you can find some on Amazon, but ... I'm paranoid about HDDs ... on Amazon they are usually declared "brand new" while they are second hand, refurbished, cleaned and polished .... sometimes not even very well and find other people's files on the HDD
...

I am paranoid, but also inconsistent, so I just ordered two Barracudas ;D
I will check them with SMART, if I find they have been used, I will return them.
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7508
  • Country: va
Re: modern storage is crazy
« Reply #5 on: April 26, 2022, 03:34:09 pm »
Quote
This morning I browsed through five different online stores, phoned to six physical ones, and none of them can sell me anything under 2Tbyte!!!

"sorry, the minimal size we have is 2Tbyte"

That can be irritating but I don't see it as unusual or crazy. The stores will want to sell stuff that shifts and not have oddball things sitting around for years waiting for the one weirdo that doesn't desire the latest and greatest, and for which they will make minuscule profit.

If you look around you'll see that there is generally a minimum price for hard disks, which is basically the lowest they will sell a disk at and still make a decent profit. Whatever the current disk size is for that price, that's the smallest disk they will sell. From what you say, that seems to be 2TB now.
 
The following users thanked this post: DiTBho

Offline ConKbot

  • Super Contributor
  • ***
  • Posts: 1400
Re: modern storage is crazy
« Reply #6 on: April 26, 2022, 03:44:45 pm »
If you can get the system to support iSCSI, then whatever is serving up the iSCSI is hardware agnostic.

And the reliability is about the same, it's just that much more data, that the non-recoverable read error rate is no longer stastically insignificant. I.e. of you put together a raid array with 8x 16tb drives, the 1 in 10^15 error rate is no longer a one in a million chance when you're doing an array rebuild after a hardware failure, hence RAID-6 being more prominent.
 

Offline SpacedCowboy

  • Frequent Contributor
  • **
  • Posts: 319
  • Country: gb
  • Aging physicist
Re: modern storage is crazy
« Reply #7 on: April 26, 2022, 04:14:40 pm »

Anyway, what I am doing is not *THE* point, but rather if those modern 14TB disks are as reliable as the smaller ones I am using.

Hard drive failures in large disks are pretty much the same as smaller ones. The failures presumably come from systemic issues to do with rotating rust at high angular velocity, rather than a weakness in the storage system itself.

Personally I don't think so; as I understand it, modern disks are less reliable, so let's fix the problem with RAID? For example, RAID-6 appears to be mandatory for that Synology 8-bay drive.

RAID6's advantage is not because larger hard drives fail more often (though it would help with that), it's because when a customer buys hard drives, they very rarely buy a single drive from multiple sources to form their drive pool, so if there is a problem in a batch of drives that makes failure more likely, it's possible for more than one of that group of drives (s)he bought to exhibit the same problem.

The scenario RAID6 helps prevent against is:
  • Lose a drive in a RAID5 array
  • Purchase new one (or have spare, hot or cold)
  • Install new drive if necessary
  • Start rebuild of the array
  • Another drive fails before the rebuild is complete

When you're rebuilding the array, you're putting far more stress on the system than during day to day use - seeks will be going back and forth over all the drives for hours on end, and *that* is not the time to find out you have another dodgy drive. So, you have 2 parity disks and even if both of them go down, you're not SOL. At that point you make a simple backup of the data as the first step, then you purchase two more disks and rebuild the array.

Note, I have to replicate the array for a second twin setup, that 1 Tbyte array was created in 2009 and is currently only 40% full, I wonder what people is expecting to store in 112TByte of space.

I have 116TB of raid at home on an Areca hardware RAID controller...

Code: [Select]
@xanadu:~$ df -h /raid
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       116T   52T   64T  45% /raid

... mounted across a 10Gbit network to all the other computers. I've used about half of it so far, with data on it going back 30 years, and it's backed up weekly to a Quantum SuperLoader 3 with 16 LTO-9 tapes (well, 15 + a cleaner tape) hosted on the same machine. One week I back up to cartridge A (8 tapes), the next week to cartridge B (7 tapes + clean) etc.

Code: [Select]
@xanadu:~$ lsscsi
[0:0:0:0]    tape    IBM      ULTRIUM-HH8      M571  /dev/st0
[0:0:0:1]    mediumx QUANTUM  UHDL             0094  /dev/sch0
[1:0:0:0]    disk    Areca    ARC-1882-VOL#000 R001  /dev/sda
[1:0:16:0]   process Areca    RAID controller  R001  -       
[N:0:2:1]    disk    Samsung SSD 960 EVO 1TB__1                 /dev/nvme0n1
[N:1:4:1]    disk    Samsung SSD 970 EVO 1TB__1                 /dev/nvme1n1
[N:2:2:1]    disk    SAMSUNG MZVLW1T0HMLH-00000__1              /dev/nvme2n1

People have different use-cases, and just because it doesn't make sense for your case, doesn't mean it's not crucial for someone else :) I would be the first to agree that I'm atypical in use-case [grin], but I'm far from the only person who runs RAID6 - in my case with an additional hot spare.
 
The following users thanked this post: DiTBho

Offline Berni

  • Super Contributor
  • ***
  • Posts: 5050
  • Country: si
Re: modern storage is crazy
« Reply #8 on: April 26, 2022, 05:20:58 pm »
Yeah disks are getting pretty big.

I guess the point is that these days making a 100GB disk is just as difficult as making a 1TB one, so they might as well make those. They probably stopped manufacturing the old bigger disk heads, so if they are using the new tiny fine heads and platters they might as well use the capacity. More dense data also makes the drive faster because one rotation brings more data bits past the head.

Id say if you want low capacity drives for use with adapters try using SSDs, they are commonly available even under 100GB and when you don't care about capacity you can more easily spring for a more reliable enterprise grade SLC flash ones.
 

Online tom66

  • Super Contributor
  • ***
  • Posts: 7334
  • Country: gb
  • Electronics Hobbyist & FPGA/Embedded Systems EE
Re: modern storage is crazy
« Reply #9 on: April 26, 2022, 05:39:51 pm »
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: modern storage is crazy
« Reply #10 on: April 26, 2022, 05:44:09 pm »
Have you looked at used drives? I haven't bought anything under 1TB in years and the only 1TB drives I have bought are SSDs. The last several hard drives I've bought are 8TB and I'd have liked to have gone larger but that's the current sweet spot. The main thing I use bulk storage for these days is video and backups. There is no market for spinning drives under 2TB anymore.
 
The following users thanked this post: DiTBho

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: modern storage is crazy
« Reply #11 on: April 26, 2022, 05:46:21 pm »
Yeah disks are getting pretty big.

I guess the point is that these days making a 100GB disk is just as difficult as making a 1TB one, so they might as well make those. They probably stopped manufacturing the old bigger disk heads, so if they are using the new tiny fine heads and platters they might as well use the capacity. More dense data also makes the drive faster because one rotation brings more data bits past the head.

Id say if you want low capacity drives for use with adapters try using SSDs, they are commonly available even under 100GB and when you don't care about capacity you can more easily spring for a more reliable enterprise grade SLC flash ones.

Most series of drives use the same platters/heads, so a 1TB version will have a single platter with one head, a 2TB will have heads on both sides of the platter, the 4TB model will have two platters and so on. The smallest drive offered will be one side of a single platter using whatever tech is used in the largest.
 
The following users thanked this post: DiTBho

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15797
  • Country: fr
Re: modern storage is crazy
« Reply #12 on: April 26, 2022, 05:50:53 pm »
We have to wonder what century the OP is living in.
 ;D
 
The following users thanked this post: tooki, DiTBho

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #13 on: April 26, 2022, 08:20:06 pm »
We have to wonder what century the OP is living in.

One year ago, I didn't have the money, but I was tempted to buy a restored Alfa Romeo 6C (1929/1930), hence I bought the dead corpse of a "Car" made in in 1873, well ... "Car" ... it's more the corpse of an horse-drawn carriage with three wheels in which someone tried to insert an experimental internal combustion engine, but which in 1885 inspired the Benz-Motorwagen, the first car to go into production with an internal combustion engine.

Talking about internal combustion, I have a ***copy (it's a copy of the copy of the copy of the original, bought on eBay) of the manifest of the first experiment done in 1826 with internal combustion of a hydrogen and oxygen mixture. Nobody believed it would work.

And next to that manifest, on the same wall of the garage where I am trying to restore (rebuild? ... many parts are rotten or missing, it's a disaster) the "motor tricycle", I have a copy of the original manifest published in 1876 for Statue of Liberty of New York, before it was installed, when news paper were excited to announce some new great stuff, the navy was transporting all pieces by ship from Europe, the locals in New York thought about how to build the base on which to install the iron-lady.

About cars and monuments, I think I live in the 19th century, while about computers, I think I live between the 20th and the 21th century, specifically around 1972 and 2009  :o :o :o
« Last Edit: April 26, 2022, 08:25:58 pm by DiTBho »
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Online mariush

  • Super Contributor
  • ***
  • Posts: 5170
  • Country: ro
  • .
Re: modern storage is crazy
« Reply #14 on: April 26, 2022, 08:28:36 pm »
What's wrong with 250 / 256/ 480 / 500 / 512 / 960 GB  SATA SSDs?

They also still make 16/32/64 GB 44 pin IDE SSDs using MLC flash memory (44 pin = ide for laptops, easily converted to regular IDE)

Not even gonna mention 4 GB / 8 GB / 16 GB / 32 GB SATA DOM SSDs for thin clients.
 
The following users thanked this post: DiTBho

Online DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4367
  • Country: gb
Re: modern storage is crazy
« Reply #15 on: April 26, 2022, 08:31:44 pm »
@SpacedCowboy
thanks, you clarified my doubts  :D
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline elekorsi

  • Regular Contributor
  • *
  • Posts: 112
  • Country: si
Re: modern storage is crazy
« Reply #16 on: April 27, 2022, 08:26:15 pm »
I was allways fascinated by the prices of Siemens Simatic memory cards. For example, list price for the 256MB (yes, megabytes) is 364€... 32GB model will cost you an arm and a leg with the price tag of 1038€. And those are active products and not some obsolete old parts...
I suspect that in the hardware they are all the same, they just make them look smaller... Oh, forgot to say, 4MB model is cheap it's only 53€  :-DD
 
The following users thanked this post: DiTBho

Offline amyk

  • Super Contributor
  • ***
  • Posts: 8526
Re: modern storage is crazy
« Reply #17 on: April 28, 2022, 12:37:58 am »
You may be able to change the HPA (Host Protected Area) size to make it appear smaller than the upper limit of the system you're putting it in.

I was allways fascinated by the prices of Siemens Simatic memory cards. For example, list price for the 256MB (yes, megabytes) is 364€... 32GB model will cost you an arm and a leg with the price tag of 1038€. And those are active products and not some obsolete old parts...
I suspect that in the hardware they are all the same, they just make them look smaller... Oh, forgot to say, 4MB model is cheap it's only 53€  :-DD
Proprietary standard means they can charge as much as they want. On the other hand, those are probably NOR flash, which is not cheap either.
« Last Edit: April 28, 2022, 12:41:41 am by amyk »
 
The following users thanked this post: DiTBho

Offline eugene

  • Frequent Contributor
  • **
  • Posts: 497
  • Country: us
Re: modern storage is crazy
« Reply #18 on: April 28, 2022, 01:45:23 am »
I suppose the OP is looking for some 4MB memory cards too.  :-DD

Just kidding around; I totally get his frustration. Ten years ago I bought a Synology NAS and stuffed two 2TB drives into it (RAID 1.) I thought 2 TB was ridiculous at the time and I was right; the drive is still only half full. The really sad thing is that the drives are 5.25". If one of them fails I'm not sure I could even find a replacement!
90% of quoted statistics are fictional
 
The following users thanked this post: DiTBho

Offline xrunner

  • Super Contributor
  • ***
  • Posts: 7836
  • Country: us
  • hp>Agilent>Keysight>???
Re: modern storage is crazy
« Reply #19 on: April 28, 2022, 01:52:29 am »
I remember I had a Radio Shack Tandy 1000 with I think a 10 or 20 MB hard drive for the thing. Man was that awesome!  :-DD
I told my friends I could teach them to be funny, but they all just laughed at me.
 
The following users thanked this post: DiTBho

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: modern storage is crazy
« Reply #20 on: April 28, 2022, 05:14:13 am »
I suppose the OP is looking for some 4MB memory cards too.  :-DD

Just kidding around; I totally get his frustration. Ten years ago I bought a Synology NAS and stuffed two 2TB drives into it (RAID 1.) I thought 2 TB was ridiculous at the time and I was right; the drive is still only half full. The really sad thing is that the drives are 5.25". If one of them fails I'm not sure I could even find a replacement!

You could replace them both with a matched set of newer drives, you could probably get 2 or even 4TB drives for next to nothing and it would be worth it just for the power savings over 5.25" drives.
 

Online tszaboo

  • Super Contributor
  • ***
  • Posts: 8218
  • Country: nl
  • Current job: ATEX product design
Re: modern storage is crazy
« Reply #21 on: April 28, 2022, 09:19:25 am »
Last time I checked few years ago, a 256GB SSD was like 30 dolaridoos. I guess precision milled metal costs more than some puny chips in a green board.
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 16385
  • Country: za
Re: modern storage is crazy
« Reply #22 on: April 28, 2022, 01:33:52 pm »
Still got a 8M Sony Memory Stick, and a 32M Nokia branded SD card. Also some 512M SD cards, and a few boxes of old IDE and SATA drives, all sub 1G, with most of them still working, at least they did pass SMART full scan with no errors. Failed ones were stripped, and went to scrap metal, and I have a nice collection of magnets on the fridge, useful for many applications, including using to operate reed switches at a distance, and holding down the GPS in the car, so that I can easily remove it.
 

Offline SpacedCowboy

  • Frequent Contributor
  • **
  • Posts: 319
  • Country: gb
  • Aging physicist
Re: modern storage is crazy
« Reply #23 on: April 28, 2022, 04:51:06 pm »
When I got my first computer, at 12, it came as components and a bare PCB. It was a Xmas present (together with a b&w, dial-tuned portable TV which I thought was awesome). After 3 months of watching TV in my bedroom, and swearing it was just as good as the one everyone shared downstairs! My parents started getting antsy about me not doing anything about the "computer" they'd bought me. We weren't well-off (my dad was a docker in Liverpool in the UK, and we lived in a 2-up/2-down terraced house in one of the less--wonderful parts of town, but they did their damndest to get me everything they thought was important, and it didn't matter to me that the TV was second-hand...)

So, off to the shed in the back yard, built it all up - young innocent me thought this was a tough thing to do, all that soldering of chips... These days it's 0201 or bust, but I digress. The thing eventually worked, and I plugged it into my TV, and typed in the famous example in the manual.

Code: [Select]
PRINT 1 + 1 = 2

Which of course printed '1' since 1+1 does equal 2. All elated, I summoned parents-mine to the bedroom to show them, and my dad took one look at the screen, grumbled "I knew it, he's buggered it" and left the room. It took me about a week to convince him that the computer was working perfectly... :-DD

Anyway, that got me hooked on computing, and I spent a year or so saving up for a *colour* one. Come next Xmas, Atari were doing a fire-sale at Dixons, and you could get a computer, with a floppy disk drive, not a tape-deck for £120. I had about £100 by then, but persuaded my nan to pitch in as that year's Xmas present. That disk drive could store 127 KB, which was *enormous*

That computer lasted me about 5 years, and just before I went to college, I bought the (at the time) state of the art Atari-ST as an upgrade. This thing had 1MB of RAM!!! Who could ever use that much ? And the floppy drive it came with could store 720 KB, and the user interface was amazing. That computer got me through college, GFA basic was (heh) basically amazing at writing simple little programs to for example simulate heat dispersing through a cube (I was reading Physics) and the black-and-white "hi-res" monitor that gave you 640x480 was really sharp and precise. 

About a year in, I bought my first ever hard disk, squandering a fair chunk of my student grant and consigning myself to ramen for a few months - it had a whopping 10MB on it. I could have gone for the cheaper 5MB one, but by this point I was savvy, so I knew that was just a trap and bought the seemingly-too-large one instead. I still remember using Signum to create lab-reports, and although it took about 30 minutes to print, and made a hell of a racket on the dot-matrix printer for the entire time, I remain convinced that my "pretty-as-hell" lab-reports contributed significantly to my overall lab grade. I took as much lab as I could...

So yeah, as I sit here using my Mac Studio Ultra on three 4K monitors, with enormous storage on tap over a network-speed I could never have dreamed of, things (and not just storage) have indeed changed :)
 

Offline eugene

  • Frequent Contributor
  • **
  • Posts: 497
  • Country: us
Re: modern storage is crazy
« Reply #24 on: April 28, 2022, 05:02:33 pm »
Would there be any point in the drive manufacturers offering built in RAID? Instead 10TB it might be 2TB with built in RAID 6. It's true you would not be able to swap a single platter if there's a failure, but it would allow you to recover all of the data and put it on a new unit.

Wouldn't help if most failures are controller related. There's probably a solution for that too, but it might be a subject for a different thread....
90% of quoted statistics are fictional
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf