Author Topic: OT: Backing up video files long term  (Read 34950 times)

0 Members and 1 Guest are viewing this topic.

Offline magetoo

  • Frequent Contributor
  • **
  • Posts: 284
  • Country: se
Re: OT: Backing up video files long term
« Reply #50 on: July 05, 2014, 10:45:26 am »
I design those things for a living. None of what you are saying is true.

Do you have a reference you could point to?

Asking because I've been hearing the exact same things that mojo-chan is saying: hard drives nowadays work not because they are error free, but because error correction is so good that the drive can cope with the inevitable faults; that those faults are a part of normal operation; that reading through a drive's sectors is necessary to spot "bits that are getting weak" (hand waving furiously) so that the controller can do its remapping magic, etc.

Would be nice to have something more to go on than "well, a guy on a forum said".
 

Offline magetoo

  • Frequent Contributor
  • **
  • Posts: 284
  • Country: se
Re: OT: Backing up video files long term
« Reply #51 on: July 05, 2014, 11:15:35 am »
For published episodes, maybe archive.org could be an option?  I know many people seem to be using them for video podcasts.

Since you have a ton of them, I would think that there might be some way to solve the problem of getting the bits there other than uploading (like mailing drives).  Their FAQ says to contact them, but doesn't go into details.
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 31246
  • Country: au
    • EEVblog
Re: OT: Backing up video files long term
« Reply #52 on: July 05, 2014, 11:22:30 am »
For published episodes, maybe archive.org could be an option?

Never looked into it.
Although isn't that public only?
I consider my raw video files private, and not something I'd generally publicly release as a whole.
 

Offline magetoo

  • Frequent Contributor
  • **
  • Posts: 284
  • Country: se
Re: OT: Backing up video files long term
« Reply #53 on: July 05, 2014, 11:48:08 am »
Yes, it's not an option for everything.  But for things that are public anyway, the EEVblog episodes, it would be free backup.

(And we viewers get another place to get them from.)


Edit: But maybe I've misunderstood what you mean.  I guess the finished product is small compared to the raw video, so maybe that's not so relevant.
« Last Edit: July 05, 2014, 12:01:33 pm by magetoo »
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 31246
  • Country: au
    • EEVblog
Re: OT: Backing up video files long term
« Reply #54 on: July 05, 2014, 12:45:44 pm »
Edit: But maybe I've misunderstood what you mean.  I guess the finished product is small compared to the raw video, so maybe that's not so relevant.

Yes, finished and uploaded video is maybe 1/5 the size of the original raw files.
 

Offline SeanB

  • Super Contributor
  • ***
  • Posts: 15379
  • Country: za
Re: OT: Backing up video files long term
« Reply #55 on: July 05, 2014, 02:27:53 pm »
I design those things for a living. None of what you are saying is true.

Do you have a reference you could point to?

Asking because I've been hearing the exact same things that mojo-chan is saying: hard drives nowadays work not because they are error free, but because error correction is so good that the drive can cope with the inevitable faults; that those faults are a part of normal operation; that reading through a drive's sectors is necessary to spot "bits that are getting weak" (hand waving furiously) so that the controller can do its remapping magic, etc.

Would be nice to have something more to go on than "well, a guy on a forum said".

Yes, it is good to refresh the drive info by doing a read every so often. SMART can do this if you ask for a full surface scan, and will take a few hours to run this, and will read all the data on the drive and see if it is all readable after error correction. If the error bursts get too big ( drive firmware dependant) for a sector the drive will then rewrite the corrected data and try again. If it decided the sector is no longer reliable it will use one of the spare sectors it has and add the bad sector to the bad block table and write the data elsewhere.

You can use SMART to do this or use other tools, the most advertised is Spinrite from GRC.com which can recover most data from bad blocks and try to get a last good read from failing bits by statistical analysis of bad bits in marginal cases. It also will do the same for failing MLC and SLC drives where the data slicer gives a bad reading.

If you ever have used Norton Disk Doctor to recover data off a flexible disk, or used TIP to recover a ZIP disk ( I have used it a lot and it works well on ZIP drives with poor sectors to recover the data) you have used Steve Gibson's software invention.
 

Offline SL4P

  • Super Contributor
  • ***
  • Posts: 2147
  • Country: au
  • There's more value if you figure it out yourself!
Re: OT: Backing up video files long term
« Reply #56 on: July 05, 2014, 02:41:05 pm »
Someone should mention that any dynamically writeable media (disk / tape / USB etc), that is visible as a logical volume - is still fair game for malware attacks.

The only 'archive' is not mounted, and the only 'safe archive' is rotated through offsite repositories durong the refresh/resync cycle.
Don't ask a question if you aren't willing to listen to the answer.
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 5130
  • Country: de
  • A qualified hobbyist ;)
Re: OT: Backing up video files long term
« Reply #57 on: July 05, 2014, 02:53:16 pm »
The problem is each nas seems to use its own thing. There are rarely updates to the firmware , some have bugs, some use reiser, so e use ext 2 , so e ext3 , some have journaling on, others off.
Nothing but trouble. Some even store the runtime image on the drive.

Actually, the various filesystems aren't a big problem. If the mount command can't detect the filesystem automatically you simply try all possible filesystems (read-only of course). And for the linux extended filesystems you got several choices to get your data. ext3 is ext2 plus journaling, i.e. you can mount an ext3 fs as ext2 read-only and read the files. It's also possible to recover a linux based RAID 5 if the disks are still ok. But that's not a task for the average user.

Quote
Usb box, format drive in ntfs and off you go. All computers can read ntfs these days. No need to screw around with obscure things .

Linux users prefer ext4 ;)
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19392
  • Country: nl
    • NCT Developments
Re: OT: Backing up video files long term
« Reply #58 on: July 05, 2014, 03:23:26 pm »
Same with people advocating hard drives. Any kind of physical failure and you lose everything. Electrical failure might be recoverable but good luck finding an identical controller in 5 years time.
If that is your opinion you have not been reading too well. The consensus it to keep a backup on multiple hard drives with 24/7 health monitoring. If one fails you replace it and you gradually upgrade to new technology. A bluray disc isn't redundant and if a new technology becomes availabe you will have issues finding a backwards compatible reader.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10942
  • Country: us
  • DavidH
Re: OT: Backing up video files long term
« Reply #59 on: July 05, 2014, 04:22:10 pm »
As I already said, it's magneto optical rather than chemical so does not degrade in the same way as older DVD and CD recordable discs.

BluRay is not magneto-optical; it does not use any magnetic layer, does not use the Kerr effect for reading, and has no bias electromagnet.  It uses a spin-coated dye or pigment or a sputtered alloy or composite just like a recordable or rewritable DVD.
 

Offline free_electron

  • Super Contributor
  • ***
  • Posts: 7378
  • Country: us
    • SiliconValleyGarage
Re: OT: Backing up video files long term
« Reply #60 on: July 05, 2014, 05:06:18 pm »
-earth magnetic field have no impact. You can put a permanent magnet on a harddisk case while it is running. No harm will happen to the data.
You don't even have to put a magnet on your hard drive. If you disassemble one, you will end up with one of the strongest permanent magnets you have in the house. So strong, you could seriously hurt yourself with one if you got pinched between it and a piece of ferrous metal.

You can find all sorts of complete twaddle on the interweb. caveat lector
The field of that magnet is trapped in the horseshoe around the vcm coil.. But you are right. Those magnets are the strongest there are. In fact, they are so strong that , if you place two of them near each other , they will 'jump' towards each other and, on impact, will shatter. The speed and force of impact is such that the magnets crack and chips and slivers fly off. This is very dangerous. Always wear safety goggles when dealing with these suckers.
Professional Electron Wrangler.
Any comments, or points of view expressed, are my own and not endorsed , induced or compensated by my employer(s).
 

Offline Monkeh

  • Super Contributor
  • ***
  • Posts: 6303
  • Country: gb
Re: OT: Backing up video files long term
« Reply #61 on: July 05, 2014, 05:32:42 pm »
You can use SMART to do this or use other tools, the most advertised is Spinrite from GRC.com which can recover most data from bad blocks and try to get a last good read from failing bits by statistical analysis of bad bits in marginal cases. It also will do the same for failing MLC and SLC drives where the data slicer gives a bad reading.

SpinRite is an archaic joke and Gibson is a prolific liar.
 

Offline free_electron

  • Super Contributor
  • ***
  • Posts: 7378
  • Country: us
    • SiliconValleyGarage
Re: OT: Backing up video files long term
« Reply #62 on: July 05, 2014, 05:33:11 pm »
Yes, it is good to refresh the drive info by doing a read every so often. SMART can do this if you ask for a full surface scan, and will take a few hours to run this, and will read all the data on the drive and see if it is all readable after error correction. If the error bursts get too big ( drive firmware dependant) for a sector the drive will then rewrite the corrected data and try again. If it decided the sector is no longer reliable it will use one of the spare sectors it has and add the bad sector to the bad block table and write the data elsewhere.
WHAT ?
There is no such thing as a data refresh. There is no point in even reading periodically.

The safest data is data stored on a harddisk that has its heads retracted off platter and powered off.
-banging on it doesnt cause mechanical damage due to headstrikes as the heads are off disk , suspended and locked
-viruses cant corrupt data as it is powered down
-power surges can't fry the electronics as it is disconnected.
-the drive is a dust free sealed environment (not like a plastic wobbly disk , exposed to degradation due to light exposure, easily scratched and full of greasy fingerprints)

There is no need for surface scanning. The drives do this when reading as part of the reading process, but only for the sectors requested by the read. It is not true that they willy-nilly access passing sectors if they have no business there. A drive access only sectors requested by the host. If it encounters a read error or seek error it will retry a number of times and then mark the sector as bad. A bad sector will not prevent subsequent read attempts. It will prevent write attempts.

There is also a big difference between logical and physical mapping. You have no idea where your data really resides on the drive. Your data is also not stored as the chain of ones and zeroes you feed the disk. The data has redundancy added to it and the packet then goes through a scrambler to prevent having more than 5 consecutive ones or zeroes. One byte you feed is stored as 10 bit on the media.
Throw in the servo wedges and other stuff like lookup tables and you end up that a drive holding 1 terabyte of user data, actually stores about 1.3 terabyte. Your data is stored as 'symbols'. 5 bits are used to store 4. Depending on the order of symbols we can flick the symbol polarity.
Lets say we find out that two consecutive symbols end up being 01000 and 00010. This would give six consecutive seroes on the drive. We replace the second symbol with 11101 as this has the same user payload. A symbol can be stored as real or as inverted. It doesnt matter symbols are symmetrical. Additional symbols, containing redundant information are injected in the stream.
Additional techniques like viterbi encoding and maximum likelyhood algorithms can weed out even multi bit errors. The mathematics below it is very complex.

Optical disks use a similar mechanism. They actually store 11 bits for every byte you feed them... The 8 bit is expanded to 14 and then compressed to 11(overall compression reduces the 14 bit streams to an average of 11. Some packets are less compressable than others. This compression algorithm is tuned for music on cd's and video on DVD and bluray. The compressors also work on data of course but they are tuned for video or audio
Professional Electron Wrangler.
Any comments, or points of view expressed, are my own and not endorsed , induced or compensated by my employer(s).
 

Offline rob77

  • Super Contributor
  • ***
  • Posts: 1848
  • Country: sk
Re: OT: Backing up video files long term
« Reply #63 on: July 05, 2014, 05:46:48 pm »

WHAT ?
There is no such thing as a data refresh. There is no point in even reading periodically.


refresh to NEW media ! and yes it's a common practice.  so refresh to NEW harddrives, NEW blueray media, NEW tapes.

if you understood it as reading and writing back , then your understanding was wrong ;)
 

Offline free_electron

  • Super Contributor
  • ***
  • Posts: 7378
  • Country: us
    • SiliconValleyGarage
Re: OT: Backing up video files long term
« Reply #64 on: July 05, 2014, 05:49:57 pm »
You can use SMART to do this or use other tools, the most advertised is Spinrite from GRC.com which can recover most data from bad blocks and try to get a last good read from failing bits by statistical analysis of bad bits in marginal cases. It also will do the same for failing MLC and SLC drives where the data slicer gives a bad reading.

SpinRite is an archaic joke and Gibson is a prolific liar.
Spinrite works fine on very old RLL drives (the ones that needed dedicated controller cards with two flatcables). It also worked on early ide drives. His idea was fine and worked. But any drive above 300megabyte uses other techniques internally You cannot acces raw head data like you could in old drives.

Spi rite today is a good program that simp,y does multiple retries on bad sectors. Normally the drive will do a fixed number of tries , flag an error and give the datablock , errors included, to the host. And that's it. Spinrite is nothing more than a tool that, on a user level, keeps sending retry operations, grabbing the returned block , comparing them and filtering out what data is 'static' and what data is destroyed.
A block can be a few kilobyte. A damaged block is not completely corrupt. It may just be a few bits in the stream, there where the mechanical damage is(and the magnetic layer is removed). Only those bits will change from retry to retry. The rest stay static. So yes, you can recover partial data by collecting two or three retires and finding the static bits. If you are lucky and the filesystem also uses some form of error correction and redundancy you may retrieve the data. If there is a catastrophical failure and entire sectors, including the serve wedge, are gone there is no recovery. If the servo wedge and track data block (not the user data but the data used by the drive to find a sector) is destroyed you cannot find it. It can be retrieved on a spin stand but the drive itself does not have the capability to do that. It would cost too much to give the drive such mechanical precision. We simply employ electronic servo mechanisms to keep the head on track. If there is no indicator where the track is, the servo doesnt work. The drive does not have the mechanical precision to put it on a track and leave it there. A spinstand can do that, a drive can't.

I can go on for days explaining all the trickery and mechanisms involved. It is a very complex system, devised for maximum data safety.

I trust drives much more than optical (self recorded) media. Take a recorded dvd and let it lay in you office exposed to ambient light. Many degrade after a year or so. A lot of cheap ones use an organic material that degrades in light. So even if you keep em in the dark : when reading them the laser in the drive is exposing them....

There are long term storage grade disks havng a metal coating. But those can only be written at maximum 2 or 4 speed. I have very old recorded CD from traxdata. These are gold and you can clearly see the holes under a strong microscope. Recording those took 25 minutes for 650 megabyte. These disks are 16 or 17 years old and work fine. The last 'silver' one i wrote from emtec (basf) was corrupted after 3 months... Simply because it lay in the open.
Gimme a nice cast aluminum frame as enclosure any time.

But, keep three copies and store them in different places.
Professional Electron Wrangler.
Any comments, or points of view expressed, are my own and not endorsed , induced or compensated by my employer(s).
 

Offline XOIIO

  • Super Contributor
  • ***
  • Posts: 1592
  • Country: ca
Re: OT: Backing up video files long term
« Reply #65 on: July 05, 2014, 05:55:55 pm »

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10942
  • Country: us
  • DavidH
Re: OT: Backing up video files long term
« Reply #66 on: July 05, 2014, 05:56:26 pm »
There is no need for surface scanning. The drives do this when reading as part of the reading process, but only for the sectors requested by the read. It is not true that they willy-nilly access passing sectors if they have no business there. A drive access only sectors requested by the host. If it encounters a read error or seek error it will retry a number of times and then mark the sector as bad. A bad sector will not prevent subsequent read attempts. It will prevent write attempts.

RAID controllers often include a scrubbing option which periodically reads an entire drive allowing the drive controller or the RAID controller to reallocate marginal or bad sectors.  Sectors may be reallocated on read even when the data is good or at least recovered.

Quote
There is also a big difference between logical and physical mapping.

There is a logical to physical mapping table and at least on older drives, spare sectors are distributed over the entire surface.  High end drives that used a dedicated servo track and supported low level formatting could optionally include a spare sector on every track lowing the number of reported bad sectors but also sacrificing capacity.
 

Offline free_electron

  • Super Contributor
  • ***
  • Posts: 7378
  • Country: us
    • SiliconValleyGarage
Re: OT: Backing up video files long term
« Reply #67 on: July 05, 2014, 05:58:01 pm »

WHAT ?
There is no such thing as a data refresh. There is no point in even reading periodically.


refresh to NEW media ! and yes it's a common practice.  so refresh to NEW harddrives, NEW blueray media, NEW tapes.

if you understood it as reading and writing back , then your understanding was wrong ;)

That's not what SeanB is talking about. Copying to new media is one kind of refresh. Fine with me.
He was talki g about reading the same drive periodically as to scan the surface for damage.

Data on a drive can o ly get damaged bqy mechanical impact (headstrike). So , by reasing you create an i stance where headstrike could happen. Are you sure that the data you just read is now still intact ? After all the head passes multiple times over the same track. The data marked ok in the first pass could now be destroyed during the second pass when accessing a different sector on that track. Oopsie.. Chicken and egg problem (and no, what you think is track 1 sector 1 and track 1 sector two are not necessarily adjacent to each other, in fact they may physicall lay on different tracks ! Physical and logical are two different things. You have no control over phyqsical, inly the drive knows. This is done te maximise throughput on the drive and shorten seek operations. I formation is staggered.
Professional Electron Wrangler.
Any comments, or points of view expressed, are my own and not endorsed , induced or compensated by my employer(s).
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 19392
  • Country: nl
    • NCT Developments
Re: OT: Backing up video files long term
« Reply #68 on: July 05, 2014, 06:08:15 pm »
Yes, it is good to refresh the drive info by doing a read every so often. SMART can do this if you ask for a full surface scan, and will take a few hours to run this, and will read all the data on the drive and see if it is all readable after error correction. If the error bursts get too big ( drive firmware dependant) for a sector the drive will then rewrite the corrected data and try again. If it decided the sector is no longer reliable it will use one of the spare sectors it has and add the bad sector to the bad block table and write the data elsewhere.
WHAT ?
There is no such thing as a data refresh. There is no point in even reading periodically.

The safest data is data stored on a harddisk that has its heads retracted off platter and powered off.
-banging on it doesnt cause mechanical damage due to headstrikes as the heads are off disk , suspended and locked
-viruses cant corrupt data as it is powered down
-power surges can't fry the electronics as it is disconnected.
-the drive is a dust free sealed environment (not like a plastic wobbly disk , exposed to degradation due to light exposure, easily scratched and full of greasy fingerprints)
But you never know if it works until you power it up again. That is the big gamble besides other people grabbing the disk and use it for other purposes. And as others pointed out it may be impossible to replace the mechanical and electronical parts.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5549
  • Country: us
Re: OT: Backing up video files long term
« Reply #69 on: July 05, 2014, 06:11:07 pm »

WHAT ?
There is no such thing as a data refresh. There is no point in even reading periodically.


refresh to NEW media ! and yes it's a common practice.  so refresh to NEW harddrives, NEW blueray media, NEW tapes.

if you understood it as reading and writing back , then your understanding was wrong ;)

I agree, I've done many media refresh copies when we used tapes (the big ones like in the old movies) it was common practice to copy the media to new fresh tapes every so often (don't recall exactly how often now). Then send a copy off site in case of a fire or some other event that could destroy everything in site.

Hierarchical backup was the craze later in the mid/late 90's not sure where they are at now but they where VERY expensive. They would have hard drives, digital tapes and optical media as well. with redundant power supplies and hot swappable RAID drives, the tapes and optical had may bays and some robotics where involved to move tapes and discs around.
 

Offline free_electron

  • Super Contributor
  • ***
  • Posts: 7378
  • Country: us
    • SiliconValleyGarage
Re: OT: Backing up video files long term
« Reply #70 on: July 05, 2014, 06:20:18 pm »
There is no need for surface scanning. The drives do this when reading as part of the reading process, but only for the sectors requested by the read. It is not true that they willy-nilly access passing sectors if they have no business there. A drive access only sectors requested by the host. If it encounters a read error or seek error it will retry a number of times and then mark the sector as bad. A bad sector will not prevent subsequent read attempts. It will prevent write attempts.

RAID controllers often include a scrubbing option which periodically reads an entire drive allowing the drive controller or the RAID controller to reallocate marginal or bad sectors.  Sectors may be reallocated on read even when the data is good or at least recovered.

Quote
There is also a big difference between logical and physical mapping.

There is a logical to physical mapping table and at least on older drives, spare sectors are distributed over the entire surface.  High end drives that used a dedicated servo track and supported low level formatting could optionally include a spare sector on every track lowing the number of reported bad sectors but also sacrificing capacity.

Raid comtrollers can scrub : but the drive firmware needs to suppport it. You will need SAS drives. Your runof the mill ide or sata can't do that.

Dediated servo tracks no longer exist. That has gone away with the diskpacks. Damage to the servotrack was a catastrophy as the entire disk became unusable , true the head for servo was read only but a mechanical damage on that track was a disaster. In diskdrives as we know emfrom the PC world , servo data is interspersed with user data. That is the base operating principle of the drive.
There is a servotrack created by the formatter to lay down the servo wedges. Once the wedges have been written that servotrack is no longer accessible. It has never been accessible by the drive.

Here is how a drive is 'formatted' in the factory. You have a surface with no information on it. So how do you place markers ? The drive mechanics is not precise enough and only works if servo mechanisms can stabilise the head.

Well. Enter the formatter. An additional head is loaded on the outer rim of the drive. Speed control of the motor doesnt even work as we need to measure interval of servo wedges... Which have not been written yet.
So the formatter writes a single 'ping'. The motor controllr is told :disengage your servo , go in open loop and simply drive based on your crystal frequency.
The formatter now reads the ping and measures the time between two pings. This gives it accurate rotational speed (crystals drift enough so motor speed is off enough to have an impact on data retrieval). The formatter now tunes speed up or down by writing control register in the motor controller. Once the ping is seen at the required rate a square wave is now laid down. Every 'click' (o e to zero or zero to one) marks a potential sector change( not all tracks have the same amount of sectors. Track on the outside are longer than tracks on the inside )
We now have a precise physical marker coming from the disk that can tell the electronics : write now.

The formatter now grabs the headstack mechanically and moves it using its serve mechanism with anoptical encoder. So this is like a stepper motor with very fine precision. We walk the tracks and everytime there is a tick detected we write a servoburst and the track/sector datablock.
Ance this is done we can retract the head that was reading the ticks, close that opening, decouple the headstack and close that opening as well. The drive is now sealed and ready for surface test.
The servo mechanisms now all work : the servoburst controls both speed of the motor and timing of write and read operatins and the datablocks tells us where we are. Amplitude detectors keep us centered on the block. Now we write the remaining empty data with a pattern (not just all zeroes... We need changes there so we can do amplitude detection to find the center of the track)

Anyway. Like i said, i can go on and on about this. There are now event techniques where a formatter is no longer needed. The electronics that recovers data is so smart and adaptive that speed fluctuations are not a problem. It can adapt at will. Think of it as a serial port where the baidrate should be 9600 but can drift from 9000 to 10000 at random without data corruption. That is what the signal coming from the head looks like. The data samplers can adapt on the fly. You only need a track position but that can also be done because now we have velocity control on the voicecoil.
Professional Electron Wrangler.
Any comments, or points of view expressed, are my own and not endorsed , induced or compensated by my employer(s).
 

Offline KK

  • Regular Contributor
  • *
  • Posts: 100
  • Country: us
Re: OT: Backing up video files long term
« Reply #71 on: July 05, 2014, 08:05:22 pm »
^^^ thanks for that insight. Very interesting.
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 3938
  • Country: ro
  • .
Re: OT: Backing up video files long term
« Reply #72 on: July 05, 2014, 08:19:31 pm »
(this is going a bit off topic but I'm genuinely curious)

free_electron, I've often wondered why hard drive manufacturers wouldn't design a hard drive that would have a head stack on each corner of the unit, so four in total. Just read either four tracks at the same time or a single track using all four heads and assemble the data into the cache of the drive in a continuous segment...
I'm thinking this may allow slower rotation speeds which may improve reliability and reduce the heat generated.. or am I wrong about these?

Also, is there something physically or otherwise problematic with moving back to 5.25" format with todays technology, let's say in order to make a 10 TB hard drive using 4400-5400 rpm and several platters for companies  that want drives for storage? Is it just too hard to make such large platters or it's just considered old technology, they think it wouldn't sell? Or it would be too expensive to change factories to make it when ssd drives are gaining ground?

---

mojo_chan : hard drives fail in 3-5 years even without keeping them on 24/7.  I run my system 24/7 on and have four drives in the system, oldest is about 4 years old and works fine. I replaced one recently due to starting to develop bad sectors though.

Here's another article with statistics regarding hard drives and other hardware, from a large french retailer : http://linustechtips.com/main/topic/108284-huge-list-of-failure-rates-on-pc-components-french-but-i-translated-nearly-everything/  Point is these people don't all keep their systems on 24/7 and still have hdd failures (preemptive comment: yes, we may not know how those people that return hardware treated those drives, they could have punched the pc or dropped the drives on floor for all we know).
I agree with your on just the comment about lightning strikes or fires.. that's still a risk.
 
« Last Edit: July 05, 2014, 08:24:35 pm by mariush »
 

Offline madires

  • Super Contributor
  • ***
  • Posts: 5130
  • Country: de
  • A qualified hobbyist ;)
Re: OT: Backing up video files long term
« Reply #73 on: July 05, 2014, 08:29:55 pm »
It makes more sense to outsource the task and use cloud storage, where the data is duplicated over multiple drives in different parts of the world and you don't have to worry about any of it. Just encrypt if you are worried about that sort of thing. Even on a slow internet connection bandwidth is infinite over time.

Have you ever tried to restore 1TB of data from a cloud based backup? I would prefer to restore from a local disk (in an USB box). But the larger problem is to transfer those 1TB to three different cloud storage services. Or do you got a STM64 or better for internet access?
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 10942
  • Country: us
  • DavidH
Re: OT: Backing up video files long term
« Reply #74 on: July 05, 2014, 08:47:01 pm »
Raid comtrollers can scrub : but the drive firmware needs to suppport it. You will need SAS drives. Your runof the mill ide or sata can't do that.

SAS drives may have a specific command for it but SATA drives which will reallocate on read may be scrubbed just reading the drive.  Even my oldest SATA RAID controllers and drives support this.

Quote
Dediated servo tracks no longer exist. That has gone away with the diskpacks.

Dedicated servo designs have been gone for a long time.  I just meant to point out that back then, spare sectors were distributed across the disk surfaces to support reallocation and I am sure the same is still the case.

Quote
<stuff about embedded servo formatting>

The part I always found interesting is the economics.  Formatting and testing takes time so the only difference between two different drives with different capacities may be the time spent on the machine which writes and verifies the formatting.

Semiconductors are often the same way.  The difference between two different operational amplifiers may only be the time to test them.

 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf