Author Topic: Basic Questions About NAS  (Read 3365 times)

0 Members and 1 Guest are viewing this topic.

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Basic Questions About NAS
« on: June 17, 2023, 09:22:30 am »
I am starting the process of purchasing a NAS system for my home network. As my home only has myself and my son, who has little interest in activities that use storage, this system will be primarily be used by myself for things like mechanical and electronic design (2D and 3D CAD), PCB creation, 3D printing, family photo storage, and video production. One of my greatest concerns is protecting the files from loss so redundancy is a must. And I do not wish to use cloud storage. I will be using two or even three different computers as well as an Android cell phone on this network (which includes Wifi). I hope that is a good description of the usage this NAS will be called on to support.

I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three? This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?

A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.

I probably will have more questions, but those are the ones at the top of my mind at this point. Thanks in advance for any help on these questions.
« Last Edit: June 17, 2023, 09:24:40 am by EPAIII »
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline Shonky

  • Frequent Contributor
  • **
  • Posts: 290
  • Country: au
Re: Basic Questions About NAS
« Reply #1 on: June 17, 2023, 09:48:31 am »
RAID is not a backup....

What happens when you accidentally delete a file and it instantly replicates across all your 3 or 4 replicated disks? Or once your off site disk gets swapped in.

What happens if your house burns down? Or gets nearby lightning strike smoking your computers?

You won't use the cloud but are ok storing your almost live disk in a somewhat secured location?

Reconsider off site backup IMO. Easy enough to properly encrypt with your own keys etc.
 
The following users thanked this post: MK14

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: Basic Questions About NAS
« Reply #2 on: June 17, 2023, 01:04:45 pm »
Based on experience with humans, I’d echo some of the concerns above. Things that need a human to remember and bother to do often don’t get done and surely don’t get done as reliably as an automated computer solution.

I started with an 8-bay Synology many years ago and have been very happy with it and often recommend that (or similar) course to others. You could also go with a 4 or 5 bay if that fit your needs better.

I’d do a minimum 4-disk SHR-2 with BtrFS and either have a cold spare drive sitting on your shelf or rely on Amazon to get me one in 2 days. I’d also create a shared volume for the most critical data that has snapshots and is automatically backed up off site somewhere (your own NAS at an office/other home, a friend/family member NAS, or cloud).

I’m designing to never lose critical files over multiple decades and protect against hardware failure but also user/application error, malware, and crypto lockers.

You can also create a less critical data shared volume that lacked off-site backup to save money on data that isn’t as critical.

Under the covers, Synology is using standard Linux block and file system software, so you could recover the data without Synology if you really had to.

This sounds like shilling for them, but I have no vested interest, just a long-time very happy user. The price isn’t the cheapest, but I bought mine 8 years ago and expect to use it at least another 5 years. (I’ve just recently cycled out the drives for larger and newer drives after about 7 years of power on time. It was a clear, easy, and slow [but unattended] process.) Even if it broke today, I’d be at a per-month price that I think is a good value.
« Last Edit: June 17, 2023, 07:03:58 pm by sokoloff »
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 3597
  • Country: us
Re: Basic Questions About NAS
« Reply #3 on: June 17, 2023, 06:48:33 pm »
I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three?

All RAID systems support this, but the specifics of how to accomplish it depend on the device.  Dedicated RAID devices (like synology, qnap) will typically be configured to send you an alert when a drive fails, have an indicator to tell you which drive failed, and likely will have automatic or "single click" rebuild when you put a new drive in.  Purpose built RAID systems also almost always are designed for hot-swap.  The specifics would be in the manual.  If you went DIY, you would have to take care of all that yourself, and you might have to power down to do the swap.

Also, I think 3 drive RAID1 is probably not what you need.  In parity RAID (raid5, raid6) especially large arrays is really common to prefer double parity (raid6) to guard against drive failure during rebuild, but for a simple mirror setup (which by definition is small and relatively fast to rebuild), I think 2 drive failures are uncommon and the ones that do happen are often N drive failure caused by controller failure, power surge, getting unlucky with a HDD model that has a design flaw, theft, etc.  It's better to put that money into offsite backup.

Quote
This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?

Having a hot or cold spare is a good idea.  Rotating a "spare" in and out of the array as offsite backup is not really ideal it can work, but it is pretty limited:  First off, it's only a single backup, and your backup is invalid during the rebuild.  It's also "backing up" irrelevant data such as unused blocks.  Ideally you would like to design your backup around having 2 or more "full" backups and some number of "incremental" backups, such that when you go to reuse your oldest full backup, you still have a full backup and multiple incrementals.  For instance, you might do a full backup every 2 or 4 weeks, and incrementals nightly or a few times per week.  This gives you strong protection against hardware failure, user error, malware, and depending on how "offsite" you are talking, natural disaster.

You might not have enough network bandwidth to easily do a full backup even monthly.  In that case, you might just do on-site (but offline) backup, or use a cloud service that allows you only ever transmit incremental snapshots but maintains redundant full snapshots remotely.

Quote
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.

Most if not all consumer NAS on the market today are using relatively standard Linux or Free-BSD RAID tools.  You can likely plug them into a standard computer with a free operating system and get them to show up properly.  You would need to check on the particular system what OS, RAID, and filesystem format it uses.  A few systems used to use a proprietary format and would require you to purchase a replacement controller from the original manufacturer if you could find one, but I don't think that's common any more.
 

Offline dobsonr741

  • Frequent Contributor
  • **
  • Posts: 600
  • Country: us
Re: Basic Questions About NAS
« Reply #4 on: June 17, 2023, 07:07:12 pm »
Based on what EPAIII said, EPAIII needs a backup, not necessarily a NAS. I would seriously reconsider a cloud backup option, as the NAS comes with serious ownership burden, if you really want to recover in case of data loss/theft/house burning down.

Consider the offsite backup, that the NAS would not be solving at all.

I would do a calculation, how much I would spend on building/maintaining/upgrading/testing/tinkering with the NAS, vs the benefit of a monthly or annual subscription fee, and doing more fun in the remainder of my lifespan the unburdening of the NAS maintenance would give back.

Oh, and privacy is not need to be harmed. It’s a right, just need to choose the right service and platform to buy.
 

Offline james_s

  • Super Contributor
  • ***
  • Posts: 21611
  • Country: us
Re: Basic Questions About NAS
« Reply #5 on: June 17, 2023, 08:00:42 pm »
I think a good place to start is establish how much storage space is needed. Every PCB design I've ever made could be stored on a single CD-R, quite a few of them would fit on a floppy disk, they're just not very big. My FPGA projects take up more space but even those are not huge. I typically back up all of my projects on DVD-R media, a single disc will hold pretty much everything.
 

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Re: Basic Questions About NAS
« Reply #6 on: June 18, 2023, 06:29:11 am »
Thank you for actually addressing my questions. I do greatly appreciate that.

In summary:

Hot swap seems to be a yes, but check before buying.

Using a "spare" as a back-up seems to have issues.

Drives working in a computer also seems to be a yes, but again check before buying.


More information from me:

I do not think I will be going DIY. Too many other things on my plate. I want to buy something that is as close to plug-n-play as possible.

How much storage space is needed? At present my main computer has a 1 TB drive with about 364 or 395 GB presently used, depending on which figures you use. As I said, I intend to start making some videos, so that could increase but I am not sure just how fast. My experience with video storage needs is from professional experience and at least ten years old. I seem to remember a 1 TB drive holding between 4 and 8 or 10 hours of video, depending on the quality level. I am thinking about at least a 2 TB level, perhaps more depending on the price of drives. I would like it to be expandable to 8 TB or more by buying new drives, if possible. I do understand that if I use RAID 1, there will be no data compression beyond the original coding.

I do fully realize that I need BOTH local storage and back-ups. I like the idea of NAS because I want to be able to access the data from multiple locations in my house and shop: office, workshop, bedroom, and perhaps others. Of course, being just one person using it, it would be accessed from only ONE location/device at a time.

I was really hoping I could combine my storage needs with my back-up needs. Let me try to lay out a thought once more and see how many holes you can punch in it.

1. I buy a three or four slot NAS.
2. I buy four drives, each 2 TB or more - perhaps just three initially but shortly later the fourth one.
3. I keep at least two drives in the NAS at all times. One more is off site in a bank box at all times. And one can be en-route between locations or in the process of being updated. I may modify that initially if I can only afford three drives at first.
4. While the third drive is being updated IN THE NAS so no network bandwidth seems to be needed, there are two working drives that both contain all my data and a fourth one that is only a week old, at worst.

Worst case seems to be if the NAS and three drives go up in smoke and can't be salvaged. I still have the fourth one in the bank vault. I buy a new NAS and three more drives and am back in business with no data loss. That is no worse than depending on a single drive as I mostly do now. And what's the chance of the NAS AND three drives all going bad at the same time?

On the other hand, seems like you guys are going to force me to think about the cloud. I need to look into how much 1 or 2 TB of cloud storage would cost. Any suggestions as to where to look? The good? The bad? The ugly?

One more question. I seem to have seen that, at least some NAS systems have a USB port where an external drive can be attached. Is this for a back-up drive? If so, should I look into that possibility? Again it seems that no network bandwidth would be used or needed.



I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three?

All RAID systems support this, but the specifics of how to accomplish it depend on the device.  Dedicated RAID devices (like synology, qnap) will typically be configured to send you an alert when a drive fails, have an indicator to tell you which drive failed, and likely will have automatic or "single click" rebuild when you put a new drive in.  Purpose built RAID systems also almost always are designed for hot-swap.  The specifics would be in the manual.  If you went DIY, you would have to take care of all that yourself, and you might have to power down to do the swap.

Also, I think 3 drive RAID1 is probably not what you need.  In parity RAID (raid5, raid6) especially large arrays is really common to prefer double parity (raid6) to guard against drive failure during rebuild, but for a simple mirror setup (which by definition is small and relatively fast to rebuild), I think 2 drive failures are uncommon and the ones that do happen are often N drive failure caused by controller failure, power surge, getting unlucky with a HDD model that has a design flaw, theft, etc.  It's better to put that money into offsite backup.

Quote
This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?

Having a hot or cold spare is a good idea.  Rotating a "spare" in and out of the array as offsite backup is not really ideal it can work, but it is pretty limited:  First off, it's only a single backup, and your backup is invalid during the rebuild.  It's also "backing up" irrelevant data such as unused blocks.  Ideally you would like to design your backup around having 2 or more "full" backups and some number of "incremental" backups, such that when you go to reuse your oldest full backup, you still have a full backup and multiple incrementals.  For instance, you might do a full backup every 2 or 4 weeks, and incrementals nightly or a few times per week.  This gives you strong protection against hardware failure, user error, malware, and depending on how "offsite" you are talking, natural disaster.

You might not have enough network bandwidth to easily do a full backup even monthly.  In that case, you might just do on-site (but offline) backup, or use a cloud service that allows you only ever transmit incremental snapshots but maintains redundant full snapshots remotely.

Quote
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.

Most if not all consumer NAS on the market today are using relatively standard Linux or Free-BSD RAID tools.  You can likely plug them into a standard computer with a free operating system and get them to show up properly.  You would need to check on the particular system what OS, RAID, and filesystem format it uses.  A few systems used to use a proprietary format and would require you to purchase a replacement controller from the original manufacturer if you could find one, but I don't think that's common any more.
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Re: Basic Questions About NAS
« Reply #7 on: June 18, 2023, 06:34:52 am »
Err, YES. Just about every set of PCB files could fit on a floppy. But designing PCBs is ONLY ONE of the things I will be doing.

Trust me when I say that videos for YouTube will take a LOT more space. A LOT MORE!

And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.



I think a good place to start is establish how much storage space is needed. Every PCB design I've ever made could be stored on a single CD-R, quite a few of them would fit on a floppy disk, they're just not very big. My FPGA projects take up more space but even those are not huge. I typically back up all of my projects on DVD-R media, a single disc will hold pretty much everything.
« Last Edit: June 18, 2023, 06:40:11 am by EPAIII »
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline Kjelt

  • Super Contributor
  • ***
  • Posts: 6458
  • Country: nl
Re: Basic Questions About NAS
« Reply #8 on: June 18, 2023, 07:50:24 am »
I would recommend a new 2 or 4 slot Synology plus a cheaper second hand 2 slot Synology (newer than 2015) as primary backup and if you don't want the cloud use a HDD hot swap slot on a home PC, a single HDD where you once every month backup the most critical data you never want to loose and store that hdd with friends, or at work as I do.
The dual Synology makes backups a whizz (hyperbackup) if you leave both running 24/7 you can automate it, I myself want to save energy so I make a backup every weekend.
If you have Synology you can use the SHR filesystem but you can also use Raid with a redundant drive.
The benefit of SHR only counts if you have more slots than used so in the future you can accomodate larger hdd's.

There is a lot to be talked about, personally I swear with Synology because:
- best software and easy to install and setup
- they are the only brand that has security at the highest priority, automatic security updates and till now it is the only brand that had no succesfull malware attack. All said I still recommend to keepyour NAS private, do not go to the accessible from the internet path because that opens up a can of worms.

The disadvantages of Synology are that they lag behind with hardware, they still have 1Gb NICS some have the opportunity to go to 10Gb but that will cost you. They are more expensive and esp. the new pro NASses(xs series) have vendor lock in on hdds and ssds.
So not all is peachy still I have five of them now my oldest from 2014 never had an issue and I use them often from different platforms (iPad, iPhone (webinterface and dedicated iOS apps from Synology), PC(webinterface & SMB), mediaplayers (SMB), TV).
Last april I switched my previous 24/7 NAS (DS216+) to a newer one (DS1522+) and to cooy the software and up and running again took me 1 hour to setup and copying was than the amount of data devided by the maximum ethernet transfer speed of the data over the NIC. In practice I was up and running in half a day, I kept my DS216 for another month  if anything went wrong, now it has another job.

Than last recommendation, if you have the NAS running 24/7 please use datacenter hdd's. Those are designed for it and have a long lifetime reliability. I switched and never looked back.

Also if you want to do extra things esp VM's for instance to run HomeAssistant, than you need a more powerfull NAS. Then buy one with 8GB RAM and a good processor.
If not I would recommend a 2 or 4 slot of the plus series.
« Last Edit: June 18, 2023, 08:05:09 am by Kjelt »
 

Online RoGeorge

  • Super Contributor
  • ***
  • Posts: 6050
  • Country: ro
Re: Basic Questions About NAS
« Reply #9 on: June 18, 2023, 08:48:31 am »
Buying dedicated NAS hardware is easier to use, but will lock you with a specific vendor.  They tend to be expensive, too.  After some years the manufacturer will drop the software/hardware support for your device and you'll be all by yourself, because their solutions are usually very specific and not open.

Saying this after buying years ago a very expensive back then WD ShareSpace (4 disks RAID).  Still works, but it was always slow, and can not update it any more.  Inside has a dedicated ARM SBC.  No option to upgrade the disks either.



My advice is to buy/assemble a standard PC box with as many disks as you want, then install FreeNAS on it (based on OpenZFS and FreeBSD).

It can be administered remote (no monitor/keyb/mouse needed), it is high performance, open source, scalable, and you are not locked to a single vendor/support.

Online beanflying

  • Super Contributor
  • ***
  • Posts: 7337
  • Country: au
  • Toys so very many Toys.
Re: Basic Questions About NAS
« Reply #10 on: June 18, 2023, 09:18:32 am »
While DIY can be done - DO NOT do it unless you want a job/project.

A few years ago I got a Synology DS218J (2 drive consumer NAS) because I wanted an idiotproof system that just worked. Add to that an additional external HDD for monthly backups which then get stored remotely and I am 'comfortable' my Data is safe. Part of the backup strategy while not perfect is Synology allows local two way syncing of select data to one (or more) of your PC's so on my main PC I sync everything that is not Media (downloaded music and video) to it. This maintains effectively a redundant copy on the NAS because of the mirroring I setup, a 'local' copy (different building) on my main PC and a third on a HDD stored remotely. Look up 3 2 1 strat and this is 'ok' as a solution.

The key here is the software and simplicity/flexibility of use over ANY of the DIY options (freenas or UNRaid are two of them).
Coffee, Food, R/C and electronics nerd in no particular order. Also CNC wannabe, 3D printer and Laser Cutter Junkie and just don't mention my TEA addiction....
 

Offline Shonky

  • Frequent Contributor
  • **
  • Posts: 290
  • Country: au
Re: Basic Questions About NAS
« Reply #11 on: June 18, 2023, 10:01:37 am »

And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.
One manually created copy doesn't count IMO and I already explained part of why. How often do you intend to manually swap disks in and out to achieve this?

Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.

You asked for advice and noted that you're inexperienced. You're being given it but it seems you have your own ideas already.
« Last Edit: June 18, 2023, 10:03:50 am by Shonky »
 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 16389
  • Country: us
  • DavidH
Re: Basic Questions About NAS
« Reply #12 on: June 18, 2023, 04:29:06 pm »
I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three? This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?

I have not used any NAS solutions, including Areca, Windows Storage Spaces, or TrueNAS which supported that.  I think TrueNAS supports replication on a second system, but I only just started playing with it.

For my critical stuff, I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary.  This allows me to grab them if I need to leave on short notice.

Quote
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.

The three systems I have used, Areca hardware RAID, Windows Storage Spaces, and TrueNAS allow transplanting of drive sets between systems.  Of course ARECA requires a duplicate HBA (host bus adapter) but I have several spares purchased through Ebay.

Areca and TrueNAS are high performance.  Windows Storage Spaces is slow, but can be good enough for Gigabit Ethernet.
 

Offline abeyer

  • Regular Contributor
  • *
  • Posts: 244
  • Country: us
Re: Basic Questions About NAS
« Reply #13 on: June 18, 2023, 08:15:16 pm »
Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.

Also, even just weekly puts a lot of extra wear on the disks. Recovering a disk from a degraded state is an intensive operation that puts a lot of stress on disks. It is a common failure mode in raid arrays to have one disk removed and a new one put in to recover... only to have a cascading failure of one or more of the "good" disks as they are put under load doing the recovery.

If you're considering this, you'd want to at least go for the "datacenter" grade disks that tend to have longer expected lifetime/cycles, and avoid having all of your disks in a single array be of the same manufacturer/model/lot.

And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.

Do you already have one you use and trust? US banks are largely trying to get out of that business, so they're getting hard to find, and more expensive even when you can. Also the history of loss/theft/seizure might not be as good as one would expect... trusted family, friend, or even a lockbox in your work office or similar might be arguably more secure.
 

Online Jeroen3

  • Super Contributor
  • ***
  • Posts: 4035
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: Basic Questions About NAS
« Reply #14 on: June 19, 2023, 07:34:49 am »
I have a Synology DS620slim. 4x1TB raid 10 + 2x500GB ssd cache. On it I run Synology Drive for cloud folder on pc's, and hyper backup.
At my parents I have a DS218 with 2x4TB SHR running hyper backup server and some camera's for their house.
Two mikrotiks arrange vpn tunneling.

I also have Synology C2 backup for €60 a year. Because you need backups at three places.

For a home nas that isn't your hobby, synology is the way. Maybe qnap, looks like they're growing a lot.
Other brands just lack in software support and leave you with a vulnerable unit in a few years.

The hardware isn't magic, everybody can do that. The software the maintaining it is where you find the value.
 

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Re: Basic Questions About NAS
« Reply #15 on: June 19, 2023, 08:57:08 am »
My own ideas. Well, yes I do have them.

But my mind is not closed. I am looking for the best way for me to do this - at a reasonable cost.

One thing I am starting to see in this discussion is you can't really trust any technology for the long run. Perhaps what I really want is several different ways of backing up in case one or two of them become obsolete. But don't know if I can afford too much of that. I was looking at some old files tonight and saw that I have stuff that goes back at least 20 years and I haven't lost it yet. I have some floppies around with even older stuff, most of which is probably impossible to access. Perhaps I am over-thinking this. The data seems to last longer than the means to access it.

Back to your concerns about a NAS not being back-ups. You seem to be thinking I am talking about just ONE extra drive. But I am not. I could have as many extra drives as money to throw at it. With two drives permanently in the NAS, there could be two, three, even four or more in that bank box or being updated in the NAS. If I did that once a week then I would have back-ups that are 0 weeks, 1 week, 2 weeks, and perhaps even more weeks old. That should be more than enough for my purposes. I only said that I may START with three drives.

But, I am also interested in the USB connected drives that have been mentioned.

I am not decided on anything. I am LOOKING at different ways of doing this and trying to see the pluses and minuses of each of them.

Do you have any definite system in mind? One that would give me the advantages of a NAS as well as a good back-up scheme. And which will not be unsupported five or ten years from now.




And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.
One manually created copy doesn't count IMO and I already explained part of why. How often do you intend to manually swap disks in and out to achieve this?

Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.

You asked for advice and noted that you're inexperienced. You're being given it but it seems you have your own ideas already.
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Re: Basic Questions About NAS
« Reply #16 on: June 19, 2023, 09:05:43 am »
Interesting thoughts here. You say, "I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary."

By USB to SATA I assume you mean a free standing SATA drive being attached to a USB jack on a computer or NAS box or whatever. And where would that batch file be, in the computer? Running in the background? So the computer would need to stay on while this is taking place.

And beyond that you seem to be saying that multiple storage systems are the answer to failure in any one of them.



I am not a computer professional. So I must ask some basic questions that no one seems to discuss when describing these systems. I am thinking of using three drives in a RAID 1 configuration. That way, all of the data will be on each of the disks. If one goes bad, I still have redundancy (disks two and three) while ordering and installing a replacement drive. The first thing I want to know is can I (hot or not) swap a fourth drive into this RAID 1 and have it automatically become a fourth image of the other three? This assumes a four bay system or at least three. Then I could keep that fourth drive in a safe place, like a bank box, as an off site back-up, swapping it with one of the other three once a week or so. And if so, would this be a universal feature of any NAS? Or is it a question I should ask before buying?

I have not used any NAS solutions, including Areca, Windows Storage Spaces, or TrueNAS which supported that.  I think TrueNAS supports replication on a second system, but I only just started playing with it.

For my critical stuff, I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary.  This allows me to grab them if I need to leave on short notice.

Quote
A second, related question would be what I could do if the NAS itself goes bad, perhaps five or more years from now. The drives would have been set up/formatted by that NAS. Things in the computer world change rapidly. So there is no guarantee that that NAS or a compatible one would still be available. Even the company which made it could evaporate. So, could the individual drives be read by any computer? It seems to me that this is a very basic question that any NAS user should know the answer to.

The three systems I have used, Areca hardware RAID, Windows Storage Spaces, and TrueNAS allow transplanting of drive sets between systems.  Of course ARECA requires a duplicate HBA (host bus adapter) but I have several spares purchased through Ebay.

Areca and TrueNAS are high performance.  Windows Storage Spaces is slow, but can be good enough for Gigabit Ethernet.
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline EPAIIITopic starter

  • Frequent Contributor
  • **
  • Posts: 979
  • Country: us
Re: Basic Questions About NAS
« Reply #17 on: June 19, 2023, 09:23:03 am »
Sounds like you have had some bad experiences with drives failing at just the wrong time. Better grade of hard drives is required. That is also a common suggestion in this NAS business. I think I am being convinced of that.

But then, would doing incremental backups on a regular basis, perhaps daily and only doing full backups on, perhaps a monthly basis be a better way to go? And does the software supplied with NAS systems provide for this? Or do I need to obtain additional software for that? Or are we back to the cloud and/or data syncing?

Different manufacturers, models, or lot numbers for the hard drives. Now there is a great idea. Does this work with all NAS systems or do I need to ask? Any possible drawbacks to it?

As for banks disappearing, not in my neighborhood. I have been doing business with my present bank, which is the closest to me, for around 11 years. They underwent a name change, but are still there. Another bank, which is also within a few blocks of my home, is presently offering me $300 if I open a checking account with them. That is in progress. And there are at least two others within a mile. So, I don't think I will run out of banks anytime soon.



Say you're doing it weekly. A 1:1 copy means at best you can go back one week. As soon as you do your manual disk swap your backup is gone. So at best you have a copy from last week. At worst you have no *backup* at all - just a live copy.

Also, even just weekly puts a lot of extra wear on the disks. Recovering a disk from a degraded state is an intensive operation that puts a lot of stress on disks. It is a common failure mode in raid arrays to have one disk removed and a new one put in to recover... only to have a cascading failure of one or more of the "good" disks as they are put under load doing the recovery.

If you're considering this, you'd want to at least go for the "datacenter" grade disks that tend to have longer expected lifetime/cycles, and avoid having all of your disks in a single array be of the same manufacturer/model/lot.

And Shonky, I DID mention off site storage (bank vault) in my original post. I plan to keep one full back-up there at all times.

Do you already have one you use and trust? US banks are largely trying to get out of that business, so they're getting hard to find, and more expensive even when you can. Also the history of loss/theft/seizure might not be as good as one would expect... trusted family, friend, or even a lockbox in your work office or similar might be arguably more secure.
Paul A.  -   SE Texas
And if you look REAL close at an analog signal,
You will find that it has discrete steps.
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: Basic Questions About NAS
« Reply #18 on: June 19, 2023, 10:10:09 am »
One thing I am starting to see in this discussion is you can't really trust any technology for the long run.
In my experience, technology is more reliable than humans, especially if you put a little bit of design care upfront.


My biggest recommendation is: make sure all the backups are completely hands-off automated. My family has lost some theoretically backed-up data twice in 35 years. Once was my error during transferring from my college account to my own PC (ftp'd a zip file in text mode and detected it far too late) and the other was my Dad's hard-drive crash where the backup strategy was "every week or so, put in a DVD-R and double-click on this batch file on your desktop." When the drive failure happened years later, the most recent backup was over 30 weeks old.

Perhaps what I really want is several different ways of backing up in case one or two of them become obsolete.
Any of the commercial off-the-shelf NAS vendors I wouldn't worry about this. All of the ones I've seen (and definitely Synology) are using standard Linux (or BSD, which is similar) storage capabilities. You aren't going to unexpectedly have a storage means go obsolete and fail in a window where you can't easily get access at your data.
But don't know if I can afford too much of that.
That's often the problem in engineering. If you have a concrete budget and storage volume size in mind, sharing it might help people guide you to solutions that might fit. If you could spend $X now, $Y/month, and periodically add $Z in purchases, we could also suggest a "buy this now to start, in m months, add this to get a better level of data durability/availability".

I was looking at some old files tonight and saw that I have stuff that goes back at least 20 years and I haven't lost it yet. I have some floppies around with even older stuff, most of which is probably impossible to access. Perhaps I am over-thinking this. The data seems to last longer than the means to access it.
<snip for brevity>
Do you have any definite system in mind? One that would give me the advantages of a NAS as well as a good back-up scheme. And which will not be unsupported five or ten years from now.
If you can swing it, I'd get a 8 bay Synology NAS (DS1821+) or 5 bay (DS1522+) and put 4 drives in it in SHR-2 (which can survive two drive failures). If that's a little too rich, get a used Synology on Ebay (DS916+ is a little over $300 used) and put 4 drives in it. With SHR-2, you lose the useful capacity of 2 of the drives in service of redundancy. You can migrate to new drives [one at a time] over time, which I just completed this spring in my unit and it was easy.

Create two shares on that: one for critical (3-2-1) data and one for less critical (3-2) data.

For the critical data, plan to have it on your computer, on your NAS, and backed up offsite. Backed up offsite could be Google drive, Dropbox, a friend's NAS, whatever. That's the data that you'd really be frustrated to lose.

For less critical data, you might plan to just have it in two places, one of them being the NAS.

All of that is predicated on someone who wants to pay a little more and do a little less tinkering/learning. There is a cheaper path available that looks something like "get an old $100 desktop, put 4 drives in it, install unRAID, Proxmox or TrueNAS Core or TrueNAS Scale, and build the same type of functionality from more raw components". I'm in IT, started down that path, and then plunked down for the nicely packaged finished product and rely on that for the core of my storage needs here. unRAID and TrueNAS offer similar levels of "nicely polished" on the software side while you cobble together the parts on the hardware side. You can save quite a bit of cash upfront this way, but will likely have higher power bills month after month.

tl;dr:
flush with cash? Buy two Synology units, one large and one your-choice. Put the large one at your place and other one someplace offsite. Configure some (all?) data to replicate off-site. Consider backing up some to Google drive, backblaze, Dropbox or other cloud.

less flush? Buy one Synology and use a cloud vendor for the off-site third copy.

less? Buy a used Synology and do as previous

less and you're willing to have a minor side-hobby? Look into TrueNAS or unRAID.

less and you actively want a side-hobby? Look into proxmox

Sounds like you have had some bad experiences with drives failing at just the wrong time. Better grade of hard drives is required. That is also a common suggestion in this NAS business. I think I am being convinced of that.
I wouldn't buy bottom of the barrel drives, but I'm not entirely convinced that "datacenter grade" is needed in spinning hard drives for home users on a budget. I'd much rather design around "things break all the time, make sure the data is resilient" than to pin my hopes on better hardware failing less often. (That's a good additional layer if you can afford it, but isn't the primary.) I'd trust a two-drive redundant array of commercial quality disks than a one-drive redundant array of datacenter-grade disks. The real takeaway for me is "one drive redundancy has the possibility to screw you if a fault goes undetected on drive X and then drive Y fails". That's why two drive redundancy isn't overkill, even though it seems like it at first. Data scrubbing is also important and something that I have set to run automatically every month (built-in feature of most systems).

 

Online David Hess

  • Super Contributor
  • ***
  • Posts: 16389
  • Country: us
  • DavidH
Re: Basic Questions About NAS
« Reply #19 on: June 19, 2023, 03:16:39 pm »
Interesting thoughts here. You say, "I have a pairs of USB to SATA enclosures with 2TB SSDs and a batch file to backup the primary to the secondary."

By USB to SATA I assume you mean a free standing SATA drive being attached to a USB jack on a computer or NAS box or whatever. And where would that batch file be, in the computer? Running in the background? So the computer would need to stay on while this is taking place.

I keep the batch file on the drive, so it gets backed up also, and the batch file is just a robocopy command which copies only changed files, and removes files that no longer exist while ignoring the recycle bin and system volume information.  With USB 3 backup times are normally short and other factors limit performance.  I tried this before with USB 2 and even 15 years ago it was too slow.

Code: [Select]
robocopy q:\ r:\ /MIR /J /MT:1 /XD "$RECYCLE.BIN" "System Volume Information"
pause

I keep the backup drive deliberately offline and periodically connect it and then run the batch file manually.  I have been using Sabrent EC-UK3B enclosures because of their rugged construction, with Crucial MX500 2TB SATA drives because of their specified power loss protection.  (1) I also use Hashcheck to verify that the content of the files on both drives are identical, and since this forces reading of the contents of every file, it forces the SSDs to perform scrub on read if necessary.

This system both keeps an offline backup, and I can grab the drive on short notice if necessary like if there is an evacuation, and a single drive (or both, or all of them) are small enough to keep with my laptop.  My laptop's internal Crucial MX500 2TB SATA drive is backed up onto another Sabrent EC-UK3B enclosure in the same way. (2)

Quote
And beyond that you seem to be saying that multiple storage systems are the answer to failure in any one of them.

I suggest that none of the three systems (TrueNAS, Windows Storage Spaces, Areca RAID HBA) that I have experience with are good options for you.

I am working toward migrating to using TrueNAS for bulk storage, (3) which I think can do everything you require, however it requires more administration than you want.  Based on what others have said here, QNAS or Synology are likely your best options, but I have no experience with them.  I have an old Netgear ReadyNAS and do not recommend Netgear at all based on my experience with it and Netgear's poor support.

Using a mirror with 3 drives makes sense but it is not something I have tried yet.  What I may eventually have is a pair of TrueNAS systems with 6 to 8 drive RAIDZ2 arrays, with one replicating the other for backup purposes.  I briefly considered a TrueNAS system with 2.5" drives for portability, but high capacity 2.5" drives now all use SMR which is not suitable for TrueNAS.

(1) Samsung is very careful to associate power loss protection only with their enterprise SSDs, so I am unclear if their consumer drives have it.  Crucial is very specific about their BX500 and MX500 SSDs.  Unfortunately there are no recent tests of which modern SSDs are vulnerable to corruption from power loss.

(2) My laptop boots from its original 500MB NVMe drive which is not currently backed up.  I may replace it with a larger NVMe drive before I figure out how I want to handle it.  It has a separate available bay for a 2.5" SATA drive so I added a Crucial MX500 2TB SATA drive because why not?

(3) I am testing TrueNAS on my 12 year old Phenom II workstation hardware which only has 8GB of ECC memory making it marginal for Windows.  So far it works great and provides enough performance to saturate my 3 x 1G network using multichannel SMB3.  I need to buy a case, drives, and a HBA (host bus adapter) to complete it.
 

Offline abeyer

  • Regular Contributor
  • *
  • Posts: 244
  • Country: us
Re: Basic Questions About NAS
« Reply #20 on: June 19, 2023, 07:15:31 pm »
As for banks disappearing, not in my neighborhood. I have been doing business with my present bank, which is the closest to me, for around 11 years. They underwent a name change, but are still there. Another bank, which is also within a few blocks of my home, is presently offering me $300 if I open a checking account with them. That is in progress. And there are at least two others within a mile. So, I don't think I will run out of banks anytime soon.

Reread what I said... it's not that banks are going away, it's that they don't want to continue providing safe deposit boxes because it's expensive, risky, and low margin. Very few new branches built have them at all, and many older ones are ceasing to offer them.
 

Offline abeyer

  • Regular Contributor
  • *
  • Posts: 244
  • Country: us
Re: Basic Questions About NAS
« Reply #21 on: June 19, 2023, 07:22:47 pm »
I wouldn't buy bottom of the barrel drives, but I'm not entirely convinced that "datacenter grade" is needed in spinning hard drives for home users on a budget.

In general, sure... but specifically for the scenario where you're intentionally removing a disk from an array and swapping it with another one for offsite storage, you'll be routinely putting the array into a degraded mode and recovering from that, which can put a lot of extra load on the disks.  Can you get away without it, sure... but I suspect you'll see more frequent failures, which are ultimately more frequent points of possible loss of data if something goes wrong and you lose multiple copies at once.
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: Basic Questions About NAS
« Reply #22 on: June 19, 2023, 07:58:24 pm »
Totally fair.

Equally fair is probably “don’t do that” rather than “buy (sometimes much) more expensive drives so you can do that with impunity”.

WD Gold Datacenter 8TB: $276: WD8002FRYZ

WD Red Plus 8 TB: $150: WD80EFZZ

WD Blue 8TB: $110: WD8002FRYZ
« Last Edit: June 19, 2023, 08:06:53 pm by sokoloff »
 

Offline abeyer

  • Regular Contributor
  • *
  • Posts: 244
  • Country: us
Re: Basic Questions About NAS
« Reply #23 on: June 19, 2023, 08:32:46 pm »
Totally fair.

Equally fair is probably “don’t do that” rather than “buy (sometimes much) more expensive drives so you can do that with impunity”.

Absolutely. I guess I should have been more explicit that this is "if you must" advice, given that OP seems kind of set on DIYing their own hardware and rotation. Personally, I feel like reliable backups at the bottom of the stack and reliable monitoring/alerting at the top of the stack are the two parts of running software services that are clearly worth paying to make someone else's responsibility.
 

Online beanflying

  • Super Contributor
  • ***
  • Posts: 7337
  • Country: au
  • Toys so very many Toys.
Re: Basic Questions About NAS
« Reply #24 on: June 20, 2023, 12:43:23 am »
Seriously some completely OTT advice above here for 'home use' and as far as I can see NO actual data quantity to be stored. Get an 8 bay NAS, Enterprise storage neither of those are 'basic advice' they are advice that Rolls Royce (with Gold plated trim) or you are slumming it  :bullshit:

Things like this and even Hot swap in a home situation are overkill and that is before you get to how much power you will finish up consuming.

Lets also consider the jet engine whine of Data Center drive speeds while we are at it in a home use case.

Just running a consumption test on my NAS and NVR consumption to put them on a 12V UPS/Battery partly for data reasons and also physical security ones the NVR runs its own Wifi independant of the main network and if I am part way into. Both of these are running between 400-500W a day total. Power consumption on an 8 Bay NAS heavier processor and enterprise drives long before you get near a home brew PC based solution is what exactly?
« Last Edit: June 20, 2023, 12:44:59 am by beanflying »
Coffee, Food, R/C and electronics nerd in no particular order. Also CNC wannabe, 3D printer and Laser Cutter Junkie and just don't mention my TEA addiction....
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf