You might lose all your data in 2016

[free_electron]

Yes.  CDs and DVDs have a shelf life.  I seem to remember it's around 10 years.
I stopped using them for backups.  Instead, I use an old computer with a large drive as my main backup and backup all my data to it.  The computer for backup is ONLY used for backups and is NOT turned on other than to do a backup.  In addition, I use a USB drive as a secondary backup.

The risk of 3 hard drives failing all at the same time is slim and easier than a bunch of CDs or DVDs.

Depends. i have traxdata brand CD's (gold coating , not that greenish coating) that were written using a Kodak PCD 225 burner at double speed. These disks were written in 1996 .. still read flawlessly today.  That is 20 years ago !

usb drives are a disaster. one esd zap and kiss all goodbye.

backups : one original + one copy is NOT a backup. one original + two copies is half a backup. one original + three copies, of which one is stored in a physically different location , is a backup.

and for people using those NAS boxes. what if the NAS itself dies. what are your chances of popping the drive into a different nas and being able to read ? ( in case your particular nas machine is discontinued )

[hendorog]

There are a few people mentioning RAID 5 - just beware as there is a nasty trap for young players with RAID:

* Replacing a drive after a drive fails can make things much worse. Google Punctured RAID array if you are interested.
* Make sure you backup your backups

Regarding backup devices, I have had good experiences with a couple of QNAP NAS devices.
They backup themselves to Amazon Glacier which is a very cheap cloud service for large quantities of data.

Other services which might be of interest:
SpiderOak for important files
Gmail for email
Timemachine for OSX
Github for source code

[kripton2035]

and for people using those NAS boxes. what if the NAS itself dies. what are your chances of popping the drive into a different nas and being able to read ? ( in case your particular nas machine is discontinued )

almost none except if you have open nas OS but I don't know of one for "boxes" only for pc tranformed into NAS

[Kjelt]

There are a few people mentioning RAID 5 - just beware as there is a nasty trap for young players with RAID:

* Replacing a drive after a drive fails can make things much worse. Google Punctured RAID array if you are interested.
* Make sure you backup your backups 

I am far from a RAID specialist but reading this made me think about what a friend happened this summer:
he suffered something similar with RAID 5 but in his case it was that he had 2 RAID controllers in his server each having 4 disks so totalling 8 disks.
Now what happened (according to him) is that one disk went bad and that cause the write action that had to be distributed over all 8 disks to only go well on the 4 disks of the other controller. The controller where the disk went bad stopped all writing actions to all the disks due to the error on the one disk. So after one disk died, he had 4 disks that were up to date, 3 disks that were not up to date and 1 dead disk. He replaced the dead disk but the system could not be rebuild because there were 4 disks out of sync. Unbelievable because this is exactly what RAID 5 should protect you against 

[SpidersWeb]

RAID is fantastic for keeping your commercial server operating 24/7, but never use it as an alternative to backups.

What's really fun is when one drive actually fails, and then a few minutes later one of your cheap desktop class hard drives falls off the array because of a pointless error (which it could've resolved by itself), and then you spend 5-6 hours trying to rebuild your data with forensic/recovery software! Good times!

Good video/experiments on the subject (well, I liked it):

[hendorog]

Hehe yep good war stories

Another consideration with RAID is that you have x times the hardware and so a failure of some sort is more likely.


 

[eugenenine]

This reminds me of a practice followed at my first employ to address the issue: "Is your backup readable?"

And if you want to really test out you 'backups' - try this exercise ....

1. Get yourself a computer system ready to use - but with completely blank drives.  That is, hardware only.
2. Consider the location your system is normally run from as having been totally destroyed by, say, fire.  (So you can't grab that 'cheat sheet' pinned to your wall.)
3. Get all your 'backups' and 'restore' to this system.

Can you do it?
Do you have all the software?  O/S?  Licences?  Do you need/have the installation discs?  Product keys?  Updates?  If you need to partition drives in a particular way, do you have the means to do it?

For bonus points, send the person who knows all the ins and outs away for a couple of days (without a mobile phone) while you do this.  In real life, they may have been hospitalised as a result of the 'fire'.


How would you go?

This is one of the reasons I can't use cloud backup, it would take weeks to restore all my data.

All open source software so no licenses to worry about.

[Shock]

It's a common misconception that RAID is for data redundancy, aside from it's original use of cheap storage it only offers availability (meaning uptime). The misconception is because RAID is used in backups and data storage and has redundancy features built in to recover volumes. So it can store backups, but is not a backup system in itself and has it's own set of quirks when it comes to recovering a hardware failure.

[Shock]

Another consideration with RAID is that you have x times the hardware and so a failure of some sort is more likely.

Yes and if the array is affected all depends on the RAID level and scale you choose. Generally the larger the array and more parity, mirrors, online/offline disks waiting in the background increases the arrays uptime and life expectancy. So while greater chance an individual disk will fail less chance of losing the whole show, which is why they are used.

[MrSlack]

Perhaps ironically as this is an electronics forum, we get more power supply failures than disk failures these days. Equally as destructive as they never seem to go out quietly.

[suicidaleggroll]

Perhaps ironically as this is an electronics forum, we get more power supply failures than disk failures these days. Equally as destructive as they never seem to go out quietly.

PSU fails are not particularly destructive per my standard. They do fail quite spectacular, but rarely they destroy devices powered by them.

One of my old HDDs begs to differ
http://thesuicidaleggroll.com/hosting/DSCN0521_resize.jpg
http://thesuicidaleggroll.com/hosting/DSCN0522_resize.jpg

I had a system running fine, then the PSU failed.  The system wouldn't turn on, so I swapped the PSU for a new one, still nothing.  So I built a new machine from scratch, got it up and running, then plugged one of the old HDDs from the failed system into the new one to try to recover the data.  The system locked up on POST, so I peaked my head around the corner of the chassis to see ~5" flames coming off of the HDD.  I yanked the power cord out of the wall and it died out pretty quickly, the pictures above were the result.

[MrSlack]

That looks toasty

Perhaps ironically as this is an electronics forum, we get more power supply failures than disk failures these days. Equally as destructive as they never seem to go out quietly.

PSU fails are not particularly destructive per my standard. They do fail quite spectacular, but rarely they destroy devices powered by them.

I've got a photo somewhere of an entirely burned out HP DL380g5 we had. PSU went up, there was an alert saying failed PSU, then the internal temp alert shut it down and proceeded to just basically melt everything plastic inside. Burned up half the cables in the rack as well. We had to remove the front (!) fan tray at the opposite end of the unit with a flat blade and a hammer and replace about half a 42U of kit that was suspect.

Then there's the nice big old 3-phase UPS we had. The 415v input exploded and set fire to half the rack as well.

Power is where it's at and where it ends.

Also the amount of MacBooks I've seen where either the left IO board catches fire, the magsafe connector catches fire or the switch mode explodes is getting ridiculous now. Seems to be 5-7 years then instant flame thrower for a lot of people.

[John_ITIC]

Setup a file server with 3 HDD's and RAID 5. Setup a backup server with another 3 HDD's and RAID 5 which replicates any changes made on the file server on a daily basis. If you are concerned about your house burning down, then setup a VPN to your buddy's house and put a file server there

Any recommendation for such a solution? Ive been looking at WDs solution but I'm sure there are better / more professional options?

[moya034]

Setup a file server with 3 HDD's and RAID 5. Setup a backup server with another 3 HDD's and RAID 5 which replicates any changes made on the file server on a daily basis. If you are concerned about your house burning down, then setup a VPN to your buddy's house and put a file server there

Any recommendation for such a solution? Ive been looking at WDs solution but I'm sure there are better / more professional options?

RAID: Hardware raid, whether your motherboard has it built in, or you install a 3rd party RAID controller in an expansion slot.

Backup Software: Can't give you much advice here. I need to start researching a good Linux solution. At work we use Symantec NetBackup, which I really like.

VPN: Site to Site OpenVPN implemented with PfSense routers at both endpoints.

[hendorog]

Setup a file server with 3 HDD's and RAID 5. Setup a backup server with another 3 HDD's and RAID 5 which replicates any changes made on the file server on a daily basis. If you are concerned about your house burning down, then setup a VPN to your buddy's house and put a file server there

Any recommendation for such a solution? Ive been looking at WDs solution but I'm sure there are better / more professional options?

I've done the live sync part of this (apart from the VPN) with two QNAP NAS devices. The docs say they can also run both a VPN server and client so it should be possible to do it just with the two off the shelf boxes. Obviously a couple of PC's can do it too, it just depends on what you feel comfortable maintaining.

Personally I would avoid RAID 5 in that situation and aim to keep it as simple and reliable as possible.

[eugenenine]

I think most just rysnc across to the backup server, thats what I do.

[Shock]

I think most just rysnc across to the backup server, thats what I do.

I'll take this any day over a corrupted 100GB proprietary backup file format that for some reason or another won't open any more.

I hope people took the hint by now that RAID is really cool, until you need to get it back.

[Brumby]

A casual note for anyone with a Raid setup - make sure you have a clear communication path between yourself and the raid controller, so when a drive fails, you know about it.

I had someone with a 2 drive external Raid box under a desk and it performed quite well.  So well, in fact, that they never knew the first drive had failed until the second one did.

It seems there was no remote notification set up and the "drive fail" light couldn't be seen unless you got down on your hands and knees.

Major problems ensued because of the archaic software design.

[Brumby]

This reminds me of a practice followed at my first employ to address the issue: "Is your backup readable?"

And if you want to really test out you 'backups' - try this exercise ....

1. Get yourself a computer system ready to use - but with completely blank drives.  That is, hardware only.
2. Consider the location your system is normally run from as having been totally destroyed by, say, fire.  (So you can't grab that 'cheat sheet' pinned to your wall.)
3. Get all your 'backups' and 'restore' to this system.

Can you do it?
Do you have all the software?  O/S?  Licences?  Do you need/have the installation discs?  Product keys?  Updates?  If you need to partition drives in a particular way, do you have the means to do it?

For bonus points, send the person who knows all the ins and outs away for a couple of days (without a mobile phone) while you do this.  In real life, they may have been hospitalised as a result of the 'fire'.


How would you go?

This is one of the reasons I can't use cloud backup, it would take weeks to restore all my data.

All open source software so no licenses to worry about.

That's one of the "discoveries" such an exercise will reveal.

[hendorog]

I think most just rysnc across to the backup server, thats what I do.

I'll take this any day over a corrupted 100GB proprietary backup file format that for some reason or another won't open any more.

I hope people took the hint by now that RAID is really cool, until you need to get it back.

Yeah exactly why I like the linux based NAS's. They are just linux and use rsync - the poster above was looking for something 'professional'.

[Brumby]

That's too absolute, IMHO. You should locally back up copyright related things, such as videos and software packages, and anyway, even if you lost them, just use your invoice to request new software medias.
For those made bu yourself, such as custom code, photos, and documents, that's what you really need to look at.
Photos can be backed up online, despite you may need days to restore all of them, who cares? You can even let them stay in cloud forever, without or with only 1 local copy.
Documents, code and other textual files should not take too much space, and backing them up is virtually at no cost.

That is a far too simplistic view - and idealistic.

 and anyway, even if you lost them, just use your invoice to request new software medias.

What if the company is out of business and there's no-one to respond to your request?  What if they do not re-issue software/product keys?  What if they do not support the old version that you have been using?

You also assume that a computer system (that's used for a business, say) can run properly on a small amount of data - which ignores the situation where an extensive database is in place.  Can you imagine DigiKey or RS Components running their online systems from a subset of their database?

In truth, the questions I have raised are valid for EVERY situation, but the ANSWERS will vary.  What seems obvious to you and works for you will almost certainly be inappropriate for the majority of others.

[Halcyon]

Setup a file server with 3 HDD's and RAID 5. Setup a backup server with another 3 HDD's and RAID 5 which replicates any changes made on the file server on a daily basis. If you are concerned about your house burning down, then setup a VPN to your buddy's house and put a file server there

Any recommendation for such a solution? Ive been looking at WDs solution but I'm sure there are better / more professional options?

RAID: Hardware raid, whether your motherboard has it built in, or you install a 3rd party RAID controller in an expansion slot.

Backup Software: Can't give you much advice here. I need to start researching a good Linux solution. At work we use Symantec NetBackup, which I really like.

VPN: Site to Site OpenVPN implemented with PfSense routers at both endpoints.

There are but I wouldn't consider on-board RAID a "proper" RAID unless it's on a proper server... Dell... HP... Supermicro etc...
RAID on consumer boards is pretty much like software masquerading as hardware RAID. As for drives, go with drives suited to the enterprise market as they are designed to be installed in hardware RAID systems. Trying to run many consumer drives on a hardware RAID will cause you issues.

As for software, I actually just use Microsoft Backup that comes free with Windows (and I backup to LTO tape). It's free and does the job, albeit with very limited features. There are some free programs around but they are pretty awful. If you expect business-grade backups, you'll have to pay.

For your everyday home user, use multiple types of media, hard disks, DVD/BD discs and also upload important data to somewhere like Dropbox or Google Drive. Check/test physical media regularly.

[MrSlack]

That looks toasty

Perhaps ironically as this is an electronics forum, we get more power supply failures than disk failures these days. Equally as destructive as they never seem to go out quietly.

PSU fails are not particularly destructive per my standard. They do fail quite spectacular, but rarely they destroy devices powered by them.

I've got a photo somewhere of an entirely burned out HP DL380g5 we had. PSU went up, there was an alert saying failed PSU, then the internal temp alert shut it down and proceeded to just basically melt everything plastic inside. Burned up half the cables in the rack as well. We had to remove the front (!) fan tray at the opposite end of the unit with a flat blade and a hammer and replace about half a 42U of kit that was suspect.

Then there's the nice big old 3-phase UPS we had. The 415v input exploded and set fire to half the rack as well.

Power is where it's at and where it ends.

Also the amount of MacBooks I've seen where either the left IO board catches fire, the magsafe connector catches fire or the switch mode explodes is getting ridiculous now. Seems to be 5-7 years then instant flame thrower for a lot of people.

I had a couple cheap PSUs went boom, and nothing bad happened to the rest of the system.
The computer I use now is working on its 3rd PSU. The first one blew up due to I set its voltage to 110V while plugged this thing in a 220V socket, no other things fried.
The second PSU is a really cheap one, not died yet, but makes noise and runs very hot, so I swapped it out.
Now the system runs on its third PSU, quietly and cool, and of course, this new PSU is way expensive than the previous ones.
I also encountered a 1U PSU failure, it just died silently, and no adverse effects beside an improper shutdown.

Probably I got better luck?

Yes and the game of data persistence is to avoid relying on luck

[Rerouter]

The only other thing i will add that is less relavant now that motherboards are seeming to just get spammed by sata ports is, keep your array sizes smaller than the maximum amount of free ports you have incase a controller card fails, had just that happen, all the data was there nice and safe on the drives, but only had 2 ports, and a 4 drive array, so had to hijack a friends computer to plug all the drives into and start the very long process of extracting the data to a new storage array,

To clarify, no longer sold controller card, and if you look hard enough you can find recovery software that can make sense of just about any way a raid array can be dreamed of,

[madires]

CD/DVD: If you want durability look for MO drives. You can get MO disks with a guaranteed life time of 30 years. Those are used for archiving medical data for example if there's a legal requirement for such long periods.

NAS with software RAID: A dead NAS is no big problem since all run linux. That means that you can connect the disks to a linux PC and recover the files. The RAID tools write some very useful management information on the disks (unless you got a NAS older than 8-10 years).

Hardware RAID: Great as long as you got support, i.e. spares.

Old stuff: Copy data from old media to current media before there aren't any drives for the old media anymore. Make sure that you can access old data with your software. Convert data into a new format if necessary.