I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three?
This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.
I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three?
All RAID systems support this, but the specifics of how to accomplish it depend on the device. Dedicated RAID devices (like synology, qnap) will typically be configured to send you an alert when a drive fails, have an indicator to tell you which drive failed, and likely will have automatic or "single click" rebuild when you put a new drive in. Purpose built RAID systems also almost always are designed for hot-swap. The specifics would be in the manual. If you went DIY, you would have to take care of all that yourself, and you might have to power down to do the swap.
Also, I think 3 drive RAID1 is probably not what you need. In parity RAID (raid5, raid6) especially large arrays is really common to prefer double parity (raid6) to guard against drive failure during rebuild, but for a simple mirror setup (which by definition is small and relatively fast to rebuild), I think 2 drive failures are uncommon and the ones that do happen are often N drive failure caused by controller failure, power surge, getting unlucky with a HDD model that has a design flaw, theft, etc. It's better to put that money into offsite backup.QuoteThis assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?
Having a hot or cold spare is a good idea. Rotating a "spare" in and out of the array as offsite backup is not really ideal it can work, but it is pretty limited: First off, it's only a single backup, and your backup is invalid during the rebuild. It's also "backing up" irrelevant data such as unused blocks. Ideally you would like to design your backup around having 2 or more "full" backups and some number of "incremental" backups, such that when you go to reuse your oldest full backup, you still have a full backup and multiple incrementals. For instance, you might do a full backup every 2 or 4 weeks, and incrementals nightly or a few times per week. This gives you strong protection against hardware failure, user error, malware, and depending on how "offsite" you are talking, natural disaster.
You might not have enough network bandwidth to easily do a full backup even monthly. In that case, you might just do on-site (but offline) backup, or use a cloud service that allows you only ever transmit incremental snapshots but maintains redundant full snapshots remotely.QuoteA second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.
Most if not all consumer NAS on the market today are using relatively standard Linux or Free-BSD RAID tools. You can likely plug them into a standard computer with a free operating system and get them to show up properly. You would need to check on the particular system what OS, RAID, and filesystem format it uses. A few systems used to use a proprietary format and would require you to purchase a replacement controller from the original manufacturer if you could find one, but I don't think that's common any more.
I think a good place to start is establish how much storage space is needed. Every PCB design I've ever made could be stored on a single CD-R, quite a few of them would fit on a floppy disk, they're just not very big. My FPGA projects take up more space but even those are not huge. I typically back up all of my projects on DVD-R media, a single disc will hold pretty much everything.
And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.
I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three? This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.
Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.
And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.
And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.One manually created copy doesn't count IMO and I already explained part of why. How often do you intend to manually swap disks in and out to achieve this?
Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.
You asked for advice and noted that you're inexperienced. You're being given it but it seems you have your own ideas already.
I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three? This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?
I have not used any NAS solutions, including Areca, Windows Storage Spaces, or TrueNAS which supported that. I think TrueNAS supports replication on a second system, but I only just started playing with it.
For my critical stuff, I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary. This allows me to grab them if I need to leave on short notice.QuoteA second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.
The three systems I have used, Areca hardware RAID, Windows Storage Spaces, and TrueNAS allow transplanting of drive sets between systems. Of course ARECA requires a duplicate HBA (host bus adapter) but I have several spares purchased through Ebay.
Areca and TrueNAS are high performance. Windows Storage Spaces is slow, but can be good enough for Gigabit Ethernet.
Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.
Also, even just weekly puts a lot of extra wear on the disks. Recovering a disk from a degraded state is an intensive operation that puts a lot of stress on disks. It is a common failure mode in raid arrays to have one disk removed and a new one put in to recover... only to have a cascading failure of one or more of the "good" disks as they are put under load doing the recovery.
If you're considering this, you'd want to at least go for the "datacenter" grade disks that tend to have longer expected lifetime/cycles, and avoid having all of your disks in a single array be of the same manufacturer/model/lot.And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.
Do you already have one you use and trust? US banks are largely trying to get out of that business, so they're getting hard to find, and more expensive even when you can. Also the history of loss/theft/seizure might not be as good as one would expect... trusted family, friend, or even a lockbox in your work office or similar might be arguably more secure.
One thing I am starting to see in this discussion is you can't really trust any technology for the long run.
Perhaps what I really want is several different ways of backing up in case one or two of them become obsolete.
But don't know if I can afford too much of that.
I was looking at some old files tonight and saw that I have stuff that goes back at least 20 years and I haven't lost it yet. I have some floppies around with even older stuff, most of which is probably impossible to access. Perhaps I am over-thinking this. The data seems to last longer than the means to access it.
<snip for brevity>
Do you have any definite system in mind? One that would give me the advantages of a NAS as well as a good back-up scheme. And which will not be unsupported five or ten years from now.
Sounds like you have had some bad experiences with drives failing at just the wrong time. Better grade of hard drives is required. That is also a common suggestion in this NAS business. I think I am being convinced of that.
Interesting thoughts here. You say, "I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary."
By USB to SATA I assume you mean a free standing SATA drive being attached to a USB jack on a computer or NAS box or whatever. And where would that batch file be, in the computer? Running in the background? So the computer would need to stay on while this is taking place.
robocopy q:\ r:\ /MIR /J /MT:1 /XD "$RECYCLE.BIN" "System Volume Information"
pause
And beyond that you seem to be saying that multiple storage systems are the answer to failure in any one of them.
As for banks disappearing, not in my neighborhood. I have been doing business with my present bank, which is the closest to me, for around 11 years. They underwent a name change, but are still there. Another bank, which is also within a few blocks of my home, is presently offering me $300 if I open a checking account with them. That is in progress. And there are at least two others within a mile. So, I don't think I will run out of banks anytime soon.
I wouldn't buy bottom of the barrel drives, but I'm not entirely convinced that "datacenter grade" is needed in spinning hard drives for home users on a budget.
Totally fair.
Equally fair is probably “don’t do that” rather than “buy (sometimes much) more expensive drives so you can do that with impunity”.