Author Topic: Replacement NAS  (Read 1920 times)

0 Members and 1 Guest are viewing this topic.

Offline Sal AmmoniacTopic starter

  • Super Contributor
  • ***
  • Posts: 1674
  • Country: us
Replacement NAS
« on: April 12, 2024, 06:01:35 pm »
I've got an old Drobo NAS here at home that is 15 years old. Now that Drobo is out of business and support is no longer available, I need to replace it with something more modern before it fails and takes my data with it.

Rather than buying another commercial NAS, I've been thinking about building a FreeBSD box, adding a bunch of disks, and configuring them as a ZFS pool. Has anyone done this? Is it a good idea?
Complexity is the number-one enemy of high-quality code.
 

Offline woofy

  • Frequent Contributor
  • **
  • Posts: 335
  • Country: gb
    • Woofys Place
Re: Replacement NAS
« Reply #1 on: April 12, 2024, 07:48:22 pm »
Yes, for the last couple of years until a few weeks ago I've been running TrueNAS Core which is a FreeBSD based NAS. I'm now running TrueNAS Scale which is Debian based.
Both have performed well with no issues.
 
The following users thanked this post: Sal Ammoniac

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #2 on: April 13, 2024, 12:33:46 am »
+1 for TrueNAS. Very simple out-of-the-box solution with enterprise features. I've been using it since the start of FreeNAS. Absolutely rock solid.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9024
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Replacement NAS
« Reply #3 on: April 13, 2024, 12:24:49 pm »
Just build up a PC running Proxmox. Doesn't have to be very fancy, I have one running on an i5-7500.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline vad

  • Frequent Contributor
  • **
  • Posts: 449
  • Country: us
Re: Replacement NAS
« Reply #4 on: April 13, 2024, 08:01:35 pm »
Another vote for TrueNAS. I’ve been running FreeNAS (the old name of the free community version) and TrueNAS Core for over seven years without any issues or data loss. I currently have two servers: one old rack mounted DIY build and another TrueNAS Mini appliance from iXsystems.

Bear in mind that ZFS, with all its great features, is intolerant to crashes, and there is a near-zero chance of recovering a corrupt ZFS pool. So make sure you back up your unique data to another device or into the cloud. If you build your own NAS, pay close attention to hardware recommendations and best practices, such as ECC RAM, UPS.
 
The following users thanked this post: SiliconWizard

Offline Messtechniker

  • Frequent Contributor
  • **
  • Posts: 782
  • Country: de
  • Old analog audio hand - No voodoo.
Re: Replacement NAS
« Reply #5 on: April 13, 2024, 08:39:02 pm »
+1 for Open Media Vault.
My secondary NAS, with the primary being a Synology NAS.
German: "Kein Backup, kein Mitleid" i.e "No backup, no sympathy"
Agilent 34465A, Siglent SDG 2042X, Hameg HMO1022, R&S HMC 8043, Peaktech 2025A, Voltcraft VC 940, M-Audio Audiophile 192, R&S Psophometer UPGR, 3 Transistor Testers, DL4JAL Transistor Curve Tracer, UT622E LCR meter
 

Offline dobsonr741

  • Frequent Contributor
  • **
  • Posts: 674
  • Country: us
Re: Replacement NAS
« Reply #6 on: April 13, 2024, 09:09:38 pm »
+1 for TrueNAS. Reliable. So many features, even Apple Time Machine backup server.
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14495
  • Country: fr
Re: Replacement NAS
« Reply #7 on: April 13, 2024, 09:50:07 pm »
My current NAS is years old and I built it from scratch around a mini-ITX motherboard, WD Red HDDs in RAID, RAID1 for the root partition (so it's mirrored over 2 of the drives) and RAID5 for data, ext4 FS. Using Centos.
It's been running solid for all these years.

My next one is probably going to be a bit different, but likely not *that* different. I'll still likely be building hardware from selected parts, and the OS is probably going to be FreeBSD this time, if just to give FreeBSD a shot.
But I'll still probably use a RAID scheme - while ZFS looks nice and modern, being rather intolerant to corruption, as vad mentioned, is not something I personally want to deal with. But that may work for some others.

I'll also probably configure it myself rather than use TrueNAS, mostly because that'll be an opportunity to discover and learn FreeBSD. But otherwise, TrueNAS is fine.
« Last Edit: April 14, 2024, 12:08:39 am by SiliconWizard »
 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #8 on: April 14, 2024, 12:04:07 am »
Another vote for TrueNAS. I’ve been running FreeNAS (the old name of the free community version) and TrueNAS Core for over seven years without any issues or data loss. I currently have two servers: one old rack mounted DIY build and another TrueNAS Mini appliance from iXsystems.

Bear in mind that ZFS, with all its great features, is intolerant to crashes, and there is a near-zero chance of recovering a corrupt ZFS pool. So make sure you back up your unique data to another device or into the cloud. If you build your own NAS, pay close attention to hardware recommendations and best practices, such as ECC RAM, UPS.

Eh, I wouldn't bother with ECC RAM personally. It offers very little benefit. Even as a user who has a server rack and enterprise gear at home, if I was to rebuild my NAS, I wouldn't bother with ECC RAM. It really doesn't matter.

As for "intolerant to crashes", I strongly disagree. I've had servers simply switch off after the UPS battery dies, with no impact to the ZFS pool. ZFS is extremely resiliant. Even in cases of the underlying OS crashing, ZFS should remain intact. Severe file system damage (ZFS or otherwise) is usually either the sign of hardware problems, or user error (or both). For example, allowing the pool to degrade to such a point that you're forced to try and restore data from a failing hard disk in order to restore your pool.

ZFS is an extremely robust and resiliant file system. Would highly recommend it, but regardless, always have a backup of your data on seperate media. I still don't consider ZFS as a form of "backup", just like traditional RAID isn't.
 

Offline nightfire

  • Frequent Contributor
  • **
  • Posts: 585
  • Country: de
Re: Replacement NAS
« Reply #9 on: April 14, 2024, 12:40:50 am »
ZFS has gotten better over the years, but still a system with a really bad crash will not be recoverable. With my last employer, we began using ZFS on Solaris practically from the beginning, and in those times they had no fsck utility available- and some time later, lots of people learned stuff the hard way, that after passing some point you had to pull the backups in case of a server crash...

This said, ZFS is as good as its operator that maintains the system- and it should get lots of memory to really leverage its strengths.
 

Offline vad

  • Frequent Contributor
  • **
  • Posts: 449
  • Country: us
Re: Replacement NAS
« Reply #10 on: April 14, 2024, 01:48:45 am »
Another vote for TrueNAS. I’ve been running FreeNAS (the old name of the free community version) and TrueNAS Core for over seven years without any issues or data loss. I currently have two servers: one old rack mounted DIY build and another TrueNAS Mini appliance from iXsystems.

Bear in mind that ZFS, with all its great features, is intolerant to crashes, and there is a near-zero chance of recovering a corrupt ZFS pool. So make sure you back up your unique data to another device or into the cloud. If you build your own NAS, pay close attention to hardware recommendations and best practices, such as ECC RAM, UPS.

Eh, I wouldn't bother with ECC RAM personally. It offers very little benefit. Even as a user who has a server rack and enterprise gear at home, if I was to rebuild my NAS, I wouldn't bother with ECC RAM. It really doesn't matter.

As for "intolerant to crashes", I strongly disagree. I've had servers simply switch off after the UPS battery dies, with no impact to the ZFS pool. ZFS is extremely resiliant. Even in cases of the underlying OS crashing, ZFS should remain intact. Severe file system damage (ZFS or otherwise) is usually either the sign of hardware problems, or user error (or both). For example, allowing the pool to degrade to such a point that you're forced to try and restore data from a failing hard disk in order to restore your pool.

ZFS is an extremely robust and resiliant file system. Would highly recommend it, but regardless, always have a backup of your data on seperate media. I still don't consider ZFS as a form of "backup", just like traditional RAID isn't.

Many decades ago, when I lived in Russia, there was a culture among drivers not to wear seatbelts. At my workplace, the chauffeurs took it as a personal affront if you dared to buckle up in the passenger seat, as if questioning their safe driving skills. They were seasoned professionals, with more years behind the wheel than they could count. They all presented a compelling argument: they claimed they had driven cars throughout their entire careers, and none of them had ever been injured in a car accident.

Can you run TrueNAS on non-recommended hardware and never suffer total loss of a ZFS pool? Sure, you can, if you're lucky enough. Should you run it on a machine with non-ECC memory and without battery power backup? That depends on many factors, such as how much you value your data, whether you have a backup, what your RTO and RPO are for your disaster recovery plan, and how much you value your time, keeping recovery time in mind.

If the NAS crashes before it syncs pending metadata changes to disks (where metadata stands for file allocation tables of the beautiful COW file system), your entire ZFS pool will be lost with little prospect of recovery. And unlike inferior NTFS, FAT32, and EXT file systems, you will not find any tool, commercial or open source, to salvage even a fraction of data from a bad ZFS pool.

But who am I to suggest you wear the seatbelts?
 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #11 on: April 14, 2024, 02:51:00 am »
If the NAS crashes before it syncs pending metadata changes to disks (where metadata stands for file allocation tables of the beautiful COW file system), your entire ZFS pool will be lost with little prospect of recovery. And unlike inferior NTFS, FAT32, and EXT file systems, you will not find any tool, commercial or open source, to salvage even a fraction of data from a bad ZFS pool.

That's the entire point of copy-on-write file systems, like ZFS. It's designed to withstand sudden power failures or incomplete writes, without catastrophic loss of data. Live, good data is never overwritten and that includes the metadata.
 

Offline vad

  • Frequent Contributor
  • **
  • Posts: 449
  • Country: us
Re: Replacement NAS
« Reply #12 on: April 14, 2024, 03:19:36 am »
If the NAS crashes before it syncs pending metadata changes to disks (where metadata stands for file allocation tables of the beautiful COW file system), your entire ZFS pool will be lost with little prospect of recovery. And unlike inferior NTFS, FAT32, and EXT file systems, you will not find any tool, commercial or open source, to salvage even a fraction of data from a bad ZFS pool.

That's the entire point of copy-on-write file systems, like ZFS. It's designed to withstand sudden power failures or incomplete writes, without catastrophic loss of data. Live, good data is never overwritten and that includes the metadata.

Design and real-world use are different things. The Boeing 737 MAX was supposed to be a superior design over the 737NG. In real life, it proved to be a disaster. Even then, Sun Microsystems never intended for ZFS to run on a recycled, ancient desktop from a dumpster.

You may refer to TrueNAS community forums for hardware guides and horror stories from those who did not RTFM.

Edit: Also, all appliances from TrueNAS vendor iXsystems come with ECC memory.
« Last Edit: April 14, 2024, 03:28:44 am by vad »
 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #13 on: April 14, 2024, 05:05:25 am »
iXsystems' hardware requirements are quite broad. See: https://www.truenas.com/docs/core/gettingstarted/corehardwareguide/
They don't recommend ECC as a requirement, but says you can use it. Since their appliances are designed for enterprise, of course they would come with ECC RAM. Most, if not all, enterprise gear does.

I appreciate your analogies vad, but I don't think you're making a very strong argument. ZFS is a resilient filesystem by design and it's certainly lived up to that design in real-world applications for quite a long time.

If you can demonstrate ZFS pool corruption on a PC by simply yanking the power cord or causing the OS to crash, I'll stand to be corrected. Use the cheapest, crappiest hardware if you like, as long as it meets their requirements. However based on many years of experience in this industry, I haven't seen it happen unless there is some underlying cause that hasn't been addressed (which I usually chop up to user error). As they say, the proof is in the pudding. If you can show me that ZFS is "intolerant to crashes" (your words, not mine), then I will withdraw my earlier statements. And by "show", I don't just mean some vague forum post, anyone can cherry pick comments on the internet, but I mean actual proper evidence of this alleged weakness in ZFS.

You've already claimed that "If the NAS crashes before it syncs pending metadata changes to disks (where metadata stands for file allocation tables of the beautiful COW file system), your entire ZFS pool will be lost with little prospect of recovery" which is demonstrably false, so I'm sorry, I don't have a lot of faith in your advice.
« Last Edit: April 14, 2024, 05:23:03 am by Halcyon »
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #14 on: April 14, 2024, 03:06:22 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.

Additionally modern nas systems have many extra features. I'm quite happy with a Qnap nas btw but that is only one data point.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Messtechniker

  • Frequent Contributor
  • **
  • Posts: 782
  • Country: de
  • Old analog audio hand - No voodoo.
Re: Replacement NAS
« Reply #15 on: April 14, 2024, 04:48:43 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.
That's precisely why I run a PC NAS once a week for a few hours as a secondary NAS.
Agilent 34465A, Siglent SDG 2042X, Hameg HMO1022, R&S HMC 8043, Peaktech 2025A, Voltcraft VC 940, M-Audio Audiophile 192, R&S Psophometer UPGR, 3 Transistor Testers, DL4JAL Transistor Curve Tracer, UT622E LCR meter
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14495
  • Country: fr
Re: Replacement NAS
« Reply #16 on: April 14, 2024, 07:56:36 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.

"Any form" is a vast range. Between a full desktop-like PC purposed for this, with an average idle power consumption of 80W-100W and a small system around a mini-ITX motherboard which will idle at around 15W-20W, there's quite a span.

Other than that, if this is a NAS for personal or small business use, it may not (in the case of personal, most surely not) require being on 24/7, but maybe only a couple hours a day. You don't need to leave it on. My personal NAS is built around a mini-ITX motherboard, draws about 20W when "idle", and I put in in standby most of the time (where it draws only 1W or so) and wake it up with WOL when needed.
 
The following users thanked this post: NiHaoMike

Offline coppice

  • Super Contributor
  • ***
  • Posts: 8657
  • Country: gb
Re: Replacement NAS
« Reply #17 on: April 14, 2024, 08:10:26 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.
You do need to be careful about power consumption, but saying all PCs are inefficient is misleading. I have have been using the same Intel i3 system as a home server for about 12 years. It idles at 12-13W from the wall, and quite a bit of that is taken by 2 3.5" hard disks. Choosing a suitable motherboard and power supply is a big factor. Some motherboards with massive overclocking potential can have a horrible idle power draw. Some power supplies seem to be designed with no regard to their efficiency outside the range laid down in the 80PLUS spec.
 
The following users thanked this post: NiHaoMike

Offline macboy

  • Super Contributor
  • ***
  • Posts: 2257
  • Country: ca
Re: Replacement NAS
« Reply #18 on: April 14, 2024, 09:26:08 pm »
Bear in mind that ZFS, with all its great features, is intolerant to crashes, and there is a near-zero chance of recovering a corrupt ZFS pool. So make sure you back up your unique data to another device or into the cloud. ...
My experience is the exact opposite. I am still using a ZFS pool that I created 15 years ago. The disks in the array are now more than an order of magnitude larger, but the pool and my data has persisted all this time. I have backups, but have never needed to recover a byte of data from them. I have replaced old small disks with new big ones, replaced sick/dying disks with healthy ones, and replaced outright dead disks with new ones. Pool always up and available, data intact.
The difference may be the operating system. I have always used Solaris or a derivative. ZFS was designed on - and for - Solaris.

I do agree though, that just using redundancy (whether ZFS or some other RAID scheme) is not equivalent to having a separate backup.
 
The following users thanked this post: Sal Ammoniac

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #19 on: April 14, 2024, 10:36:48 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.
You do need to be careful about power consumption, but saying all PCs are inefficient is misleading. I have have been using the same Intel i3 system as a home server for about 12 years. It idles at 12-13W from the wall, and quite a bit of that is taken by 2 3.5" hard disks. Choosing a suitable motherboard and power supply is a big factor. Some motherboards with massive overclocking potential can have a horrible idle power draw. Some power supplies seem to be designed with no regard to their efficiency outside the range laid down in the 80PLUS spec.
Well, compared to a relatively simple ARM processor optimised for super low power, even an i3 is quite a power hog. For comparison: the Qnap NAS I have consumes little over 7Watts max. including the hard drive  according to the specs. But I'm sure this number is too high as I put a low power (2.9W idle), 5400 rpm hard drive in it.
« Last Edit: April 14, 2024, 10:38:58 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 8657
  • Country: gb
Re: Replacement NAS
« Reply #20 on: April 14, 2024, 11:34:53 pm »
I'd keep power consumption in mind. A pc in any form isn't particularly efficient.
You do need to be careful about power consumption, but saying all PCs are inefficient is misleading. I have have been using the same Intel i3 system as a home server for about 12 years. It idles at 12-13W from the wall, and quite a bit of that is taken by 2 3.5" hard disks. Choosing a suitable motherboard and power supply is a big factor. Some motherboards with massive overclocking potential can have a horrible idle power draw. Some power supplies seem to be designed with no regard to their efficiency outside the range laid down in the 80PLUS spec.
Well, compared to a relatively simple ARM processor optimised for super low power, even an i3 is quite a power hog. For comparison: the Qnap NAS I have consumes little over 7Watts max. including the hard drive  according to the specs. But I'm sure this number is too high as I put a low power (2.9W idle), 5400 rpm hard drive in it.
I used to have power consumption like that with an Atom based home server, before the i3 one I have used since. I can't remember why I chose a higher consumption option for the replacement. There were still plenty of Atom options back then. Its sad they disappeared. The thing I like about these options is anything running on other Linux machines I have runs exactly the same on that server. Pushing functions beyond basic NAS to that server is trivial.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9024
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Replacement NAS
« Reply #21 on: April 14, 2024, 11:39:08 pm »
Well, compared to a relatively simple ARM processor optimised for super low power, even an i3 is quite a power hog. For comparison: the Qnap NAS I have consumes little over 7Watts max. including the hard drive  according to the specs. But I'm sure this number is too high as I put a low power (2.9W idle), 5400 rpm hard drive in it.
Then the vendor does something unfriendly to the user after the device goes out of support and all those savings are gone. Or worse, the device fails and the proprietary RAID effectively has ransomwared your data. At the least, make sure there's a decent aftermarket firmware community for it (even if you have no initial plans to use it) and that there's a way to read off the data by connecting the disks to a regular PC.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 
The following users thanked this post: SiliconWizard

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16627
  • Country: us
  • DavidH
Re: Replacement NAS
« Reply #22 on: April 14, 2024, 11:50:22 pm »
I repurposed my old Phenom 2 940 workstation as a NAS for testing various configurations.

TrueNAS Core (FreeBSD) worked great, but I had problems with Samba where files or directories could be created that could not be deleted.  TrueNAS Scale (Linux) worked but did not support the hardware quite as well.

Since I had a Windows 10 license for it, I ended up just using Storage Spaces for now which is more flexible about modifying the array of drives.  Performance was about the same either way, but getting good performance out of Storage Spaces requires learning how to configure details of the array from the command line.

 

Offline vad

  • Frequent Contributor
  • **
  • Posts: 449
  • Country: us
Re: Replacement NAS
« Reply #23 on: April 15, 2024, 12:26:05 am »
You've already claimed that "If the NAS crashes before it syncs pending metadata changes to disks (where metadata stands for file allocation tables of the beautiful COW file system), your entire ZFS pool will be lost with little prospect of recovery" which is demonstrably false, so I'm sorry, I don't have a lot of faith in your advice.

I don't mean to be rude, but there's an old Russian saying: “A fool learns from his own mistakes, a wise man from the mistakes of others.”

So COW file system is a superior design compared to journaling file systems. I got it. There's another file system called Btrfs. Like ZFS, it's a superior copy-on-write file system “designed to withstand sudden power failures or incomplete writes without catastrophic data loss”. Here's a recent firsthand experience shared by a prominent figure whom I've been following for almost 30 years:

Quote
Why is the Btrfs file system as implemented by Synology so fragile?
February 29, 2024 by Phil Greenspun

We had a few seconds of power loss the other day. Everything in the house, including a Windows machine using NTFS, came back to life without any issues. A Synology DS720+, however, became a useless brick, claiming to have suffered unrecoverable file system damage while the underlying two hard drives and two SSDs are in perfect condition. It’s two mirrored drives using the Btrfs file system (the Synology default, though ext4 is also available as an option). Btrfs is supposedly a journaling file system, which should make this kind of corruption impossible. Yet searching the Internet reveals that Synology suicides are commonplace. Here’s one example that pins the blame on the SSDs being enabled as read/write caches (but given that the SSDs are non-volatile why isn’t the Synology software smart enough to deal with the possibility of a power outage even when read/write caching (seems to be the default) is enabled? The Synology web page on the subject says you need two SSDs (which I have) for “fault tolerance” and doesn’t mention that the entire NAS can become a brick after losing power for a few seconds).

Given that Synology has only one job, i.e., the secure storage of data, this strikes me as a spectacular failure of corporate mission.

Readers: Have you seen this kind of failure before? NTFS was introduced by Microsoft in 1993 and I’ve never seen it completely destroyed by a power interruption.

So, perhaps ZFS is the better COW file system compared to Btrfs, and maybe all the folks on the TrueNAS forum are mistaken about ECC memory and battery backups, along with all the TrueNAS guides being wrong. But what if they're not? What if wearing seatbelts really does save your life when you hit a pole at 40 miles per hour?

The full blog post is available here: https://philip.greenspun.com/blog/2024/02/29/why-is-the-btrfs-file-system-as-implemented-by-synology-so-fragile/
 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #24 on: April 15, 2024, 01:57:02 am »
So, perhaps ZFS is the better COW file system compared to Btrfs, and maybe all the folks on the TrueNAS forum are mistaken about ECC memory and battery backups, along with all the TrueNAS guides being wrong. But what if they're not? What if wearing seatbelts really does save your life when you hit a pole at 40 miles per hour?

The full blog post is available here: https://philip.greenspun.com/blog/2024/02/29/why-is-the-btrfs-file-system-as-implemented-by-synology-so-fragile/

I have little experience with Btrfs, but what I do know is that Synology's offerings shouldn't be compared to TrueNAS. I also can't comment on how Synology implements btrfs or their RAID solution. Maybe in the past it has been a poor design? I don't know? But one thing to keep in mind is that Synology's products are aimed at the consumer and prosumer market and that includes their RackStation products (which I have used before). They are designed to be an easy to use, out-of-the-box solution for individuals and businesses that don't want to mess around with complex configurations, permissions etc... TrueNAS on the other hand is an enterprise solution, but one that can be used in the home in a scaled down version.

iXsystems have been around for a long time and along with a FreeNAS/TrueNAS have a longstanding proven track record, as does ZFS. People aren't necessarily mistaken about ECC, there are use cases for it, but unless you have money to burn, that use case isn't in the home (or even small to medium office environments). None of the TrueNAS guides are wrong either. If you have ECC, you'll get the added benefit from it, if not, it's not a deal-breaker. My advice is, don't compare one product to a entirely different product, it'll lead you to making false assumptions.

I personally use a UPS so that I don't have to login to the server after short power outages and re-input the volume encryption keys. Graceful shutdowns, whilst always recommended, are really just an afterthought for me. I sleep pretty well at night knowing my data is safe.
« Last Edit: April 15, 2024, 01:58:45 am by Halcyon »
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #25 on: April 15, 2024, 07:51:54 am »
Well, compared to a relatively simple ARM processor optimised for super low power, even an i3 is quite a power hog. For comparison: the Qnap NAS I have consumes little over 7Watts max. including the hard drive  according to the specs. But I'm sure this number is too high as I put a low power (2.9W idle), 5400 rpm hard drive in it.
Then the vendor does something unfriendly to the user after the device goes out of support and all those savings are gone. Or worse, the device fails and the proprietary RAID effectively has ransomwared your data. At the least, make sure there's a decent aftermarket firmware community for it (even if you have no initial plans to use it) and that there's a way to read off the data by connecting the disks to a regular PC.
Sorry, but this remark makes zero sense. A NAS is a device you physically own and have access to. Unless the hard drive(s) fail(s) or you lose the password, there is no way you can get locked out of your data even if the vendor ceases to exist. It is not cloud storage! And in case a NAS does fail, you just buy another one and restore the data from another backup. 99% of the NAS devices runs Linux anyway so the chance you can't access the hard drive from a Linux computer is next to zero. However, the chance the hard drive fails way before the NAS itself is close to 100% anyway.
« Last Edit: April 15, 2024, 08:49:17 am by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 4957
  • Country: si
Re: Replacement NAS
« Reply #26 on: April 15, 2024, 08:01:38 am »
Unraid user for almost a decade here. And would recommend using it at home.

Yes TrueNAS is the superior solution if you are after performance, but for home use it is often overkill.

While Unraid does cost money (but it is lifetime license that includes all future upgrades) it is an easy to use solution where you don't need to know what you are doing, while making it very difficult to have a catastrophic loss of data. Unlike real RAID setups this is just gluing disks together at the filesystem level while using parity drives to guard from disk failures. You can just throw random disks in there and they will join the pool of storage, no need to have same size drives, no need to restripe it over the new drives..etc. Gives the same level of protection as RAID, however if a RAID array degrades to a point where parity can no longer recover your data, then all of your data has been practically nuked. Yet here there is still hope, each drive in the array has its own filesystem and so is still individually perfectly readable in any linux machine, allowing you to recover whatever data is still left. Heck you can even mix and match filesystems, have some of the array disks be BTRFS, some XFS, some EXT4. Doesn't matter as long as linux can read it. You can even spin down the disks in the array that are not needed and only spin up the disk the file is on. You can also do VMs and Dockers and all that stuff that TrueNAS does.

If Unraid is so good why are more people not using it then? Well... performance
Because it is not actual RAID means you don't get the performance boost of RAID. So the reading performance is equal to the read speed of 1 HDD. Writing is even slower because it also has to read then write. This is a deal breaker for enterprise use where storage servers get seriously big. However for home use it is usually good enough. Modern HDDs are usually 150MB/s sequential read/write and so it is faster than a 1Gbit Ethernet connection anyway, so the speed doesn't matter (unless you are one of the rare few that have a 2.5G or 10G LAN). The write speed can also be improved by using a SSD cache. And if you want speed and have 10G networking you can still spend some extra money on making an all SSD NAS that will beat the pants off any regular HDD NAS.
 

Offline M0HZH

  • Regular Contributor
  • *
  • Posts: 206
  • Country: gb
    • QRPblog
Re: Replacement NAS
« Reply #27 on: April 15, 2024, 09:29:35 am »
I prefer a tried, tested & well supported off-the-shelf NAS for my critical data (both personal and for my small business), so I run a Synology as the main NAS with scheduled backups to another local Synology & to Amazon Glacier. Haven't lost a thing in 10 years and except the initial entry cost of a Synology unit, the running costs are quite low: power-efficient, low failure rate (except a few known models), good resale value.

For replaceable / less important data OpenMediaVault running on some old / low-power hardware works, but generally there are tradeoffs: time, effort, reliability, long term power usage for lower initial cost, performance, flexibility. It's not necessarily better.

I find the more advanced / custom NAS solutions (including TrueNAS) outside the scope of a typical home/SMB user.
 

Offline woofy

  • Frequent Contributor
  • **
  • Posts: 335
  • Country: gb
    • Woofys Place
Re: Replacement NAS
« Reply #28 on: April 15, 2024, 09:54:51 am »
I've got an old Drobo NAS here at home that is 15 years old. Now that Drobo is out of business and support is no longer available, I need to replace it with something more modern before it fails and takes my data with it.

Rather than buying another commercial NAS, I've been thinking about building a FreeBSD box, adding a bunch of disks, and configuring them as a ZFS pool. Has anyone done this? Is it a good idea?

I think its worth re-quoting the op's post. This is a new build replacement home NAS. TrueNAS Core is exactly what is being requested. TrueNAS and ZFS are proven technologies and despite some posts here, I can find no evidence that ZFS is unreliable. I don't count social media opinions as evidence. My own experience of TrueNAS is overwhelmingly positive. I've been running TrueNas here at home since the pandemic in 2020, almost 4 years now, and even longer at work where we have two machines running trueNAS. We've had plenty of power failures in that time but never any TrueNAS issues.

As far as raid is concerned, I wouldn't bother. There's no performance gain of any significance, this is a home NAS. Raid may be great for commercial use where a hot swap can restore the system without down time but for a home server its pointless. In any case, you still need a separate backup.

And on power, with idle consumption in the 10w region, does it matter. My own home NAS is (for the last few weeks) an N100 mini PC with a single 2TB SSD for the data. Power consumption is around 10W. Put that in perspective, its 100 hours operation per kwh, or 87.6 kwh/yr. At 7.5p/kwh that's £6.57 year! I can't even buy a couple of pints of beer for that.

Offline unseenninja

  • Contributor
  • Posts: 18
  • Country: se
Re: Replacement NAS
« Reply #29 on: April 15, 2024, 10:14:12 am »
If it wasn't for Intel insisting that ECC DRAM was an "enterprise feature" and making it impossible or unnecessarily costly to implement for consumer CPUs, it would be used in every PC.

A random bit flip caused by a cosmic ray is not the stuff of legends, they really do happen. As the size of each individual bit in a memory chip gets smaller and smaller, the chance that a bit flip might happen increases. Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality. A bit flip which corrupts data in memory before ZFS has checksummed it and written it to disk will never be detected until you discover that the file in question is corrupted. The original authors of ZFS say you should use ECC DRAM. Those guys know what they are talking about.

My TrueNAS has ECC DRAM and I wouldn't even think of building one without it. I also based it on an AMD CPU for this generation of the hardware as I didn't want to pay Intel's premium for something which is an essential feature.
 

Offline Halcyon

  • Global Moderator
  • *****
  • Posts: 5684
  • Country: au
Re: Replacement NAS
« Reply #30 on: April 15, 2024, 11:28:25 am »
Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality.

To be fair, usually when things fuck up, it's usually Microsoft's fault. I've been chasing weird and wonderful issues for weeks.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9024
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Replacement NAS
« Reply #31 on: April 15, 2024, 12:27:22 pm »
Sorry, but this remark makes zero sense. A NAS is a device you physically own and have access to. Unless the hard drive(s) fail(s) or you lose the password, there is no way you can get locked out of your data even if the vendor ceases to exist. It is not cloud storage!
https://www.bleepingcomputer.com/news/security/critical-rce-bug-in-92-000-d-link-nas-devices-now-exploited-in-attacks/
With a standard PC running an open source server distro, all you have to do is update it. With a lot of proprietary ARM systems, you're stuck with the vendor continuing support.
Quote
And in case a NAS does fail, you just buy another one and restore the data from another backup.
If you're trying to get back the data that has changed since the last run of next level backup, having to buy another NAS from the same vendor is pretty much the definition of ransomware.
Quote
99% of the NAS devices runs Linux anyway so the chance you can't access the hard drive from a Linux computer is next to zero.
Unless it uses some proprietary RAID to support "advanced features". Hence the reason to do some research to make sure a tool to read off the array with a standard PC exists.
If it wasn't for Intel insisting that ECC DRAM was an "enterprise feature" and making it impossible or unnecessarily costly to implement for consumer CPUs, it would be used in every PC.

A random bit flip caused by a cosmic ray is not the stuff of legends, they really do happen. As the size of each individual bit in a memory chip gets smaller and smaller, the chance that a bit flip might happen increases. Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality. A bit flip which corrupts data in memory before ZFS has checksummed it and written it to disk will never be detected until you discover that the file in question is corrupted. The original authors of ZFS say you should use ECC DRAM. Those guys know what they are talking about.

My TrueNAS has ECC DRAM and I wouldn't even think of building one without it. I also based it on an AMD CPU for this generation of the hardware as I didn't want to pay Intel's premium for something which is an essential feature.

I read somewhere that ECC is a standard feature of DDR5, has there been any independent verification that's actually the case for all DDR5?
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #32 on: April 15, 2024, 12:53:56 pm »
Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality.

To be fair, usually when things fuck up, it's usually Microsoft's fault. I've been chasing weird and wonderful issues for weeks.
I disagree. I have quite a bit of background in supplying reliable PCs (and making PCs reliable) and in my experience most of the problems in PCs are due to crappy hardware and / or drivers. Windows will run well for prolonged periods of time on good quality hardware.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #33 on: April 15, 2024, 01:02:35 pm »
Sorry, but this remark makes zero sense. A NAS is a device you physically own and have access to. Unless the hard drive(s) fail(s) or you lose the password, there is no way you can get locked out of your data even if the vendor ceases to exist. It is not cloud storage!
https://www.bleepingcomputer.com/news/security/critical-rce-bug-in-92-000-d-link-nas-devices-now-exploited-in-attacks/
But who is crazy enough to put a NAS on internet? I mean that in itself is a big no. And chances are there will be more security issues with your self build PC based NAS compared to an off-the-shelve product which should have a minimal attack surface to begin with. IF you need remote access to a NAS, do this via a VPN router / VPN client.
« Last Edit: April 15, 2024, 01:07:10 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16627
  • Country: us
  • DavidH
Re: Replacement NAS
« Reply #34 on: April 15, 2024, 02:31:53 pm »
If it wasn't for Intel insisting that ECC DRAM was an "enterprise feature" and making it impossible or unnecessarily costly to implement for consumer CPUs, it would be used in every PC.

The real cost is in system validation with the BIOS and operating system.  At least AMD allows it even if unsupported in most cases.  Unfortunately for whatever reason, AMD disables ECC on their CPUs that have built in graphics, at least up until recently, except for the Pro versions which are not generally available.  When I built my little server, I could have bought a Pro CPU from the Chinese grey market, but the increased cost was about the same as a cheap graphics card for a server which normally has no monitor, so I did that.

Quote
As the size of each individual bit in a memory chip gets smaller and smaller, the chance that a bit flip might happen increases.

Whether a bit is affected depends on the density of the charge rather than the amount.  The ionizing radiation strike distributes charge across a large volume, so if the bits are physically smaller, they pick up less charge.  DRAM designs have improved density by storing equal or slightly less charge in smaller volumes, so the charge density goes up for each bit and it become less susceptible.  In practice the result has been that radiation susceptibility leveled off several DRAM generations ago for a given amount of RAM, but of course system memory requirements still increased so systems do become more vulnerable, just not nearly as much as originally expected.

Quote
My TrueNAS has ECC DRAM and I wouldn't even think of building one without it. I also based it on an AMD CPU for this generation of the hardware as I didn't want to pay Intel's premium for something which is an essential feature.

The last Intel system I built for myself with ECC was a Pentium 4, which I still have.  Everything since has been AMD because of better ECC support.  I tried figuring out what I needed to build an Intel ECC system a couple years ago when I built my Ryzen workstation, and it was too complicated and questionable, and the Intel system would have doubled the cost of the motherboard.  High AMD motherboard prices became reasonable compared to even higher Intel motherboard prices.

Quote
I read somewhere that ECC is a standard feature of DDR5, has there been any independent verification that's actually the case for all DDR5?

It is, and it is not.  All DDR5 uses ECC internally to provide a limited amount of protection, but errors are only corrected when data is read out, and no scrubbing takes place.  This has to be the case because scrubbing every time that a row is opened would cost too much power.  How often rows can be opened is already limited by power concerns.

Normal DDR5 implements two 32-bit memory channels per DIMM, but ECC DDR5 implements two 40-bit memory channels per DIMM, which has nothing to do with the internal ECC protection.  I assume this means the chips will be 8-bits wide so one channel takes either 4 or 5 chips, and a single rank DIMM will use 8 or 10 chips.

But who is crazy enough to put a NAS on internet? I mean that in itself is a big no. And chances are there will be more security issues with your self build PC based NAS compared to an off-the-shelve product which should have a minimal attack surface to begin with. IF you need remote access to a NAS, do this via a VPN router / VPN client.

Some people are dumb, inexperienced, or desperate enough to expose the Remote Desktop Protocol or SMB port so that they can reach their system remotely.  A VPN is definitely the way to go, and is what I have always done in the past.
« Last Edit: April 15, 2024, 02:35:08 pm by David Hess »
 

Offline luudee

  • Frequent Contributor
  • **
  • Posts: 274
  • Country: th
Re: Replacement NAS
« Reply #35 on: April 15, 2024, 02:36:35 pm »
I have built a NAS a few years ago, I opted for a HW RAID controller.
I chose a Avago MegaRAID SAS 9361-16i, as I felt it had the most
space for future upgrades. Yes, it added about $1K to total cost, but
totally worth it in my opinion.

I also installed a Intel 10G X550T ethernet card.

I changed all FANs to Noctua high performance FANs.

Running ubuntu, not very optimized but it does it job quite well.

Attached are some pics of my monster !

Cheers,
rudi

 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16627
  • Country: us
  • DavidH
Re: Replacement NAS
« Reply #36 on: April 15, 2024, 02:58:49 pm »
I have built a NAS a few years ago, I opted for a HW RAID controller.
I chose a Avago MegaRAID SAS 9361-16i, as I felt it had the most
space for future upgrades. Yes, it added about $1K to total cost, but
totally worth it in my opinion.

I have had good results with the Areca RAID controllers that I have used.  I was originally planning on moving all of my bulk storage to a separate TrueNAS system when I built my Ryzen workstation, but instead upgraded my hardware RAID to an Areca 1680IX with 8x14TB drives in RAID6, plus a couple of 2TB RAID10 volumes.

I like being able to boot the system from the RAID controllers, hence the two 2TB RAID10 volumes, but Windows sometimes gets into a mode after updates that breaks booting from a volume that requires added drivers, so I am looking more favorably on a dumb controller and Storage Spaces because it is easier to replace the drives to increase the storage.

I tested booting and operating from 4 SATA SSDs in hardware RAID10 and it was not any faster.  It doubled the storage and added redundancy, but had other disadvantages.  The hardware RAID controllers are not fast enough to take good advantage of SSDs.
 

Offline luudee

  • Frequent Contributor
  • **
  • Posts: 274
  • Country: th
Re: Replacement NAS
« Reply #37 on: April 15, 2024, 03:03:43 pm »
I have built a NAS a few years ago, I opted for a HW RAID controller.
I chose a Avago MegaRAID SAS 9361-16i, as I felt it had the most
space for future upgrades. Yes, it added about $1K to total cost, but
totally worth it in my opinion.

I have had good results with the Areca RAID controllers that I have used.  I was originally planning on moving all of my bulk storage to a separate TrueNAS system when I built my Ryzen workstation, but instead upgraded my hardware RAID to an Areca 1680IX with 8x14TB drives in RAID6, plus a couple of 2TB RAID10 volumes.

I like being able to boot the system from the RAID controllers, hence the two 2TB RAID10 volumes, but Windows sometimes gets into a mode after updates that breaks booting from a volume that requires added drivers, so I am looking more favorably on a dumb controller and Storage Spaces because it is easier to replace the drives to increase the storage.

I tested booting and operating from 4 SATA SSDs in hardware RAID10 and it was not any faster.  It doubled the storage and added redundancy, but had other disadvantages.  The hardware RAID controllers are not fast enough to take good advantage of SSDs.

Hi David,

yeah, for that exact reason, Windows is VERBOTEN in my office !  >:D

Cheers,
rudi
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #38 on: April 15, 2024, 03:43:53 pm »
I tested booting and operating from 4 SATA SSDs in hardware RAID10 and it was not any faster.  It doubled the storage and added redundancy, but had other disadvantages.  The hardware RAID controllers are not fast enough to take good advantage of SSDs.
No surprise there  ;D . I have a 4 lane M.2 PCIe SSD in my PC. It shows a transfer rate of 1GB/s when reading. I doubt there are any cheap RAID controllers which support that kind of throughput. IMHO RAID as in having disks in parallel is only useful to increase throughput from hard drives (with spinning disks).
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline Berni

  • Super Contributor
  • ***
  • Posts: 4957
  • Country: si
Re: Replacement NAS
« Reply #39 on: April 15, 2024, 04:12:27 pm »
Yeah hardware RAID controller cards are not a very good idea anymore.

They are just another potential point of failure, and when they do fail you are in for a world of hurt getting things running again. The striping on your drives might not be just simply compatible with some other RAID card you might have laying around, so you better swap it with an identical one. Then you have to set it up correctly to recognize the array correctly again, if you do something particularly stupid and have it attempt an array rebuild with wrong configuration it might even nuke your data...etc

And what do you get for using a RAID card? Usually it is performance. However these days drives have evolved and CPUs are much more powerful, so in a lot of cases it is actually SLOWER to use a hardware RAID card. You can get very good performance from software RAID solutions these days. Just buy a simple SAS HBA card and throw a ZFS array at those drives on a modern CPU and you will be getting plenty of performance. No hardware configuration needed either, the HBA card can be replaced with any other HBA card by just sticking it in and booting the machine up, as long as the OS can find the drive it just works. All of this is performant enough to saturate a 10G connection.

If you are going for speed then go for NVME SSDs, you can get 5000MB/s from a single drive, so no RAID even needed to go fast. And if you do have a crazy 100G home LAN network then you can still RAID multiple together and actually saturate such a connection. If you can afford 100G networking then you can afford the SSDs and a server capable of pushing those bytes around fast enough.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 16627
  • Country: us
  • DavidH
Re: Replacement NAS
« Reply #40 on: April 15, 2024, 05:37:02 pm »
I tested booting and operating from 4 SATA SSDs in hardware RAID10 and it was not any faster.  It doubled the storage and added redundancy, but had other disadvantages.  The hardware RAID controllers are not fast enough to take good advantage of SSDs.

No surprise there  ;D . I have a 4 lane M.2 PCIe SSD in my PC. It shows a transfer rate of 1GB/s when reading. I doubt there are any cheap RAID controllers which support that kind of throughput. IMHO RAID as in having disks in parallel is only useful to increase throughput from hard drives (with spinning disks).

I should have said that my 20 year old hardware RAID controllers, which were fast for the time, cannot take good advantage of the speed of SSDs.  More recent hardware RAID controllers can definitely take advantage of SATA/SAS SSD speeds.  NVMe SSDs are another thing entirely.

Yeah hardware RAID controller cards are not a very good idea anymore.

They still have their place, like if you want to boot from a redundant volume.

Quote
They are just another potential point of failure, and when they do fail you are in for a world of hurt getting things running again. The striping on your drives might not be just simply compatible with some other RAID card you might have laying around, so you better swap it with an identical one. Then you have to set it up correctly to recognize the array correctly again, if you do something particularly stupid and have it attempt an array rebuild with wrong configuration it might even nuke your data...etc

I have not had any trouble moving my RAID sets between my different Areca cards, but that is one reason I like them.  I have been picking them up on Ebay for cheap, and refurbishing them for my own use.

Quote
And what do you get for using a RAID card? Usually it is performance. However these days drives have evolved and CPUs are much more powerful, so in a lot of cases it is actually SLOWER to use a hardware RAID card. You can get very good performance from software RAID solutions these days. Just buy a simple SAS HBA card and throw a ZFS array at those drives on a modern CPU and you will be getting plenty of performance. No hardware configuration needed either, the HBA card can be replaced with any other HBA card by just sticking it in and booting the machine up, as long as the OS can find the drive it just works. All of this is performant enough to saturate a 10G connection.

That is what I thought and why I ran performance tests using my old workstation.  TrueNAS was faster than Windows Storage Spaces by a little bit, but my old Areca hardware RAID controllers were faster than TrueNAS.  I only ended up using Storage Spaces because of the Samba problem that I mentioned earlier, and Storage Spaces being more flexible with swapping and upgrading drives.
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14495
  • Country: fr
Re: Replacement NAS
« Reply #41 on: April 15, 2024, 06:42:06 pm »
Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality.

To be fair, usually when things fuck up, it's usually Microsoft's fault. I've been chasing weird and wonderful issues for weeks.

Yes. The above was actually quite funny. The probability of a "bit flip" due to cosmic rays crashing your machine is thousands of times lower than a software bug.
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 26926
  • Country: nl
    • NCT Developments
Re: Replacement NAS
« Reply #42 on: April 15, 2024, 07:56:29 pm »
Most people experience bit flips as random blue screens of death and put it down to micros~1's software quality.

To be fair, usually when things fuck up, it's usually Microsoft's fault. I've been chasing weird and wonderful issues for weeks.

Yes. The above was actually quite funny. The probability of a "bit flip" due to cosmic rays crashing your machine is thousands of times lower than a software bug.
You can make fun of cosmic rays and it could be classified as far-fetched but radiation isn't the only possible problem. A poor power supply / power distribution design or slightly flaky memory will cause mysterious problems as well. When I first got my previous PC, it would trip up when doing a longwinded (30 minute) compilation run every now and then. Sometime it would compile OK, sometimes not. In the end I let memtest run with the most extensive tests for a good part of a day until it did find a memory failure. After some more runs and specifying the test and memory area, I managed to pinpoint it to a faulty memory module. After exchanging the memory module for a good one, the compilation process succeeded every time. Needless to say my current PC has ECC memory.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf