Come diss my NAS/Server idea!

[paulca]

Noctua fans rock once again.

With the enclosure running 2 drives flat out syncing for hours, the HDD temps have stabilised at 44*C +-1*C and there is significant air flow out the back of the unit now.... and it's much quieter.  The HD bearing whine and the head chatter are much louder than the fan.

Other than the audibles, I think the fan provides more pressure to force air through the narrow grilling.

EDIT: When I did this with 3 drives, no fan and no lid earlier all 3 drives hit "Too hot to hold" and reported 52*C.

[Mr. Scram]

Eh, tough choice.
I agree that an RPi is pointless for this purpose.

At the same time, I don't very much like using USB for mass storage, even 3.1 or faster especially if it's only one link serving a number of devices, and having two boxes is in any case cumbersome.

How many and how fast network interfaces are you considering?

DO NOT rely on any HW RAID driver, is in general a good recipe for disaster.

My solution has been an HPE Microserver Gen 8, the base model with a Cleleron 1610T (but a Xeon E3-1220lv2 is now crawling from China to Sweden...).

• 4×2 TB HDD (2 pairs of different WD models)
• 1×256 GB SSD
• boot device is an internal 32 GB USB key (there's an internal USB 3.0 socket and a Micro-SD one)
• 4+8 GB ECC RAM

FreeNAS and of course ZFS, volumes are mirrored and striped on the HDDs, the SSD is used for less important or temporary stuff.
Administration is reasonably simple, though it took some attempts to have it join the AD Domain.

The server comes with two 1 GB/s NICs - three if you count the one for the lights out, iLO, management.
They can easily be saturated towards my main PC (two NICs there, too), using the native SMB multichannel in Windows (no setup whatsoever needed) for about 200 MB/s of total transfer rate.

There's an internal PCIe×16 low profile expansion slot.

Epochal uptimes, never a glitch and it's relatively silent even in a living room.

Only power and two Ethernet cables are connected, as with iLO there's no need for a physical console.
IIRC correctly, power consumption is about 30W at idle (the coming Xeon sports half the TDP, so I hope to decrease even further), but I don't feel like turning it off to measure it...

Very good build quality and maintainability, small enough (about 25×25×25 cm) to sit on a bookshelf.

Can be found on Ebay for not much money, even decently equipped.

In the pictures a file copy and the HPE server, below my open-air Hyper-V server (picture is terrible)

It'd be interesting to see how much of a difference that Xeon makes for the power consumption. I tried to look it up but most people report full system numbers with varying amounts of HDDs included.

[indeterminatus]

RAID1 is fine, I think. It should also allow you to extend the storage later on without needing a complete replica of your setup to migrate the data.

Depending on the data you keep there, please also consider a propery backup strategy (if not already done so). RAID is no backup!

Also, 44°C sounds a bit high for my taste, I like my drives between 25°C and 40°C. Do you expect the strain on the disks to go down in "normal" operation? If so, you're probably fine. If not, you're probably fine too, but your drives might fail earlier. (Cannot cite the statistics, so please take this with a grain of salt; I think to remember it was Backblaze to post the drive failures and as a result of that, I think to remember that the ideal operation temperature would be between 25°C and 40°C ... but again, I fail to find the quote. If my memory does not serve me right, please, someone, correct me!)

[paulca]

I'm expecting much less activity in operation.  Right now drive 1 is copying to drive 2 (oversimplification), so both hammering.  It's peaking, 200Mbyte/s, troughing at 130.  For a spinner that's not bad for write perf.... unless that's Mbits....

Resilience.  If it all went on fire tomorrow, I'd be heart broken, but my life would function.

I want to minimise the return to full operation time for failures and add another safety net to/before backups.

EDIT:
If it was 200Mb/s 2Tb would copy in 10,000 seconds.  That's 3 hours.  Yes?  (+1 bottle wine)

[David Hess]

Below is a photo of the system I just built.  It is not quite a server but it could be.

- 4 x 4TB HGST drives with RAID6 boot, RAID10 swap, and RAID6 data volumes.  Can convert RAID6 volume to RAID5 later to increase available space if necessary.
- Spare Areca 1210 RAID HBA
- Spare Radeon R7 240
- AMD Zen 2 3100 CPU - Originally wanted one of the pro series APUs but availability was a problem and the 3100 ended up being less expensive.  Could not find 3300X for sale anywhere.
- 16GB ECC RAM - 2 DIMM slots still open to increase memory later if needed.
- Crucial MX500 for swap drive.
- Relatively cheap ASRock B450M Steel Legend motherboard.  Was originally going to get B450M Pro4 but availability was problematical.
- Silverstone case has air filtering, easy to clean filters, and positive pressure.  Drives operate at 40C.
- EVGA SuperNOVA 550 GA power supply - In retrospect, should have gotten power supply with 4 SATA connectors per lead to make wiring easier.

[rdl]

Beyond simple mirroring for backup purposes I don't really see any need for RAID at all.

[David Hess]

Beyond simple mirroring for backup purposes I don't really see any need for RAID at all.

It provides redundancy against simple failure and it increased I/O throughput over single or spanned disks.

[dnwheeler]

Beyond simple mirroring for backup purposes I don't really see any need for RAID at all.

My UnRaid server currently has 17 hard drives. With that number of drives, the odds are very high that at least one drive will fail every year. I am currently running dual parity, which allows for any two drives to fail at the same time with no data loss. If more than two drives fail at once, my losses are limited to just the drives that failed.

[PlainName]

What happens if you have a fire or the PSU puts 110V on the 5V rail?

[BravoV]

What happens if you have a fire or the PSU puts 110V on the 5V rail?

That risk is no greater than accidentally delete the partition, or infected by ransomware that locked all files in it.

RAID is NOT a backup, period

[Mr. Scram]

That risk is no greater than accidentally delete the partition, or infected by ransomware that locked all files in it.

RAID is NOT a backup, period

It is a backup but not a backup solution. You literally have a hardware backup and no more.

[Berni]

Yep RAID is not a definitive backup solution but then again even backup solutions come in various levels.

You could have a 2nd identical server sitting next to it that gets mirrored over every so often.
But what if a lightning strike takes out both? Okay then unplug the 2nd server after a backup is done
But what if the house burns down? Okay then put the 2nd server in a different building.
But what if a flood sweeps the area? Okay then put the 2nd server 300km away.
But what if a virus corrupts the data before mirroring starts? Okay then the 2nd server is bigger in storage and makes versions.
But what if a virus slowly corrupts the data? Okay make the 2nd server have even more storage to hold a year worth of history.
But what if a hacker targets both servers and blows them up? Okay then make a 3rd secret server that i manually type the address in even month to sync data to.
...
etc

RAID is just the first line of defense in the chain to defend from hardware failures. Sure maybe you never had a HDD failure in 15 years. but if you have 15 drives in that server that 15 year mean time between failures becomes just 1 year. So it becomes pretty likely that you will see some drive failure and its much nicer if you can simply replace the drive and move on with no downtime. Rather than bring the whole system down and rebuild from a backup that might be days or even weeks old, then bring it back up.

[PlainName]

So it becomes pretty likely that you will see some drive failure and its much nicer if you can simply replace the drive and move on with no downtime

Indeed, yes. It is hard to overstate how useful that is

But... for most purposes isn't a mirror fine for that? Obviously, if you have enough drives (not needed for the array per se) that you'll see one or two failures quite often, an array that can cope with more than one failure at a time would be useful. But if you have, say, four drives, a mirror will give you twice as much storage as the full thing and as close as dammit robustness to boot.

[Berni]

Yes mirror is useful when you have a small number of drives, but once you get past 3 drives the other parity based RAID methods are better because they can give more redundancy for wasting less space.

If you have 4x1TB drives you could have 2+2 mirroring to get 2TB of usable space with 1 allowed failure if you are unlucky or 2 if lucky.
If you have 4x1TB drives and use 3+1 parity you get 3TB of usable space with 1 allowed failure. Or 2+2 parity to give you the same 2TB of usable space but grantee 2 allowed failures (any 2 combination of drives)

Once you get to more drives say 10x1TB you can get 8+2 parity to have 8TB of usable space with guaranteed 2 allowed failures versus 5+5 mirror giving you 5TB of usable space with only 1 guaranteed allowed failure.

[PlainName]

OK, thanks

On reason I like the mirror is because when, say, an OS update goes wrong you can whip one drive out and keep it as the authoritative source whilst trying to fix things on the other.

[amspire]

I am a fan of the commercial NAS boxes like the Netgear Readnas 6 boxes. They are not cheap, but I have managed to pick up some cheap second hand and they tend to be very reliable and very quiet. Don't get the older Netgear boxes that do not run ReadyNAS 6.

They are a commercial distro, but they are basically a well configured Debian Linux box with SSH access. You can update via the Netgear ReadyNAS upgrades, but you can also use apt-get commands for upgrading and for installing your own Linux apps. I have often used my own scripts to do daily rsyncs to remote cloud sites or to remote NAS boxes.

Netgear uses BTRFS for the disk system and snapshotting, and mdadm for the raid. They do not use the RAID of BTRFS.

I like the low power - my 2 disk ReadyNAS  box is something 12W average and it handles power failures and reboots very well. The boxes are also very compact with excellent hot-swappable hardware. I got my current NAS box for about A$120 with a single WD 4G RED NAS drive. Had to buy a second RED drive.

Richard

[PlainName]

the Netgear Readnas 6 boxes

Have to say that if I were in the market for off-the-shelf NAS, Netgear wouldn't even get on the long list never mind the shortlist. They make some decent non-managed hardware, but anywhere firmware is involved they are terrible. No, terrible it too kind. Atrocious is more appropriate.

[Berni]

OK, thanks

On reason I like the mirror is because when, say, an OS update goes wrong you can whip one drive out and keep it as the authoritative source whilst trying to fix things on the other.

Well... RAID will most likely not help with that.

When the OS does a write to the drive that write gets committed to both disk drives once there is free disk time available to do so. So if the OS shoots its own foot it will do so on both mirrored drives at nearly the same time. The actual advantage of mirroring is that the two mirrors are just two identical normal disk drives that will boot in any PC. But any more fancy RAID arrays absolutely requires a working RAID controller to get any useful data from it.

For this case i actually use incremental backups. Using Veeam Agent ( https://www.veeam.com/windows-endpoint-server-backup-free.html ) i have a scheduled task that runs once per week, making a complete backup of my entire PC and storing that on the NAS server. To save space it creates a full backup every so often but then just makes incremental backup files every week. So with this i have the complete state of my entire PC stored for about 3 months into the past. If my PCs SSD explodes all i have to do is insert a new one, boot from Veeams USB recovery image, show it the path to the backup file on the network and about 1 hour later the PC will boot up in just the same state it was 1 week ago. Or if a randsome ware has trashed the drive i can go back 2 weeks or 3 or more. Or if i overwrote an important file then i can just go open the backup file like a ZIP archive and browse trough my entire HDD as it was 1 month ago to retrieve an old non overwritten version of a file.

[Mr. Scram]

I am a fan of the commercial NAS boxes like the Netgear Readnas 6 boxes. They are not cheap, but I have managed to pick up some cheap second hand and they tend to be very reliable and very quiet. Don't get the older Netgear boxes that do not run ReadyNAS 6.

They are a commercial distro, but they are basically a well configured Debian Linux box with SSH access. You can update via the Netgear ReadyNAS upgrades, but you can also use apt-get commands for upgrading and for installing your own Linux apps. I have often used my own scripts to do daily rsyncs to remote cloud sites or to remote NAS boxes.

Netgear uses BTRFS for the disk system and snapshotting, and mdadm for the raid. They do not use the RAID of BTRFS.

I like the low power - my 2 disk ReadyNAS  box is something 12W average and it handles power failures and reboots very well. The boxes are also very compact with excellent hot-swappable hardware. I got my current NAS box for about A$120 with a single WD 4G RED NAS drive. Had to buy a second RED drive.

Richard

12 watt is very low. Both disks should consume as much when in use. What kind of processor is in your system?

[Monkeh]

But any more fancy RAID arrays absolutely requires a working RAID controller to get any useful data from it.

And that's the great thing about software RAID, you're not tied to a specific model and firmware version of controller which may or may not still be available.

[PlainName]

I think you haven't grasped what I was getting at.

So if the OS shoots its own foot it will do so on both mirrored drives at nearly the same time.

Yes, exactly that. Or if you do a fat fingered delete/format/whatever. The point is that the system is borked,  but you might be able to fix it. Do you want to try on the one copy you have? No, you take one half of the mirror and store it somewhere safe, then you can try fixing the other half. If it works, fine, but if it doesn't you haven't made things worse.

With a setup where your data is striped over more than one drive, you can't do that. If that happened, then you'd have to revert to backups, but there are good reasons why that might be the last option.

[Monkeh]

I think you haven't grasped what I was getting at.

So if the OS shoots its own foot it will do so on both mirrored drives at nearly the same time.

Yes, exactly that. Or if you do a fat fingered delete/format/whatever. The point is that the system is borked,  but you might be able to fix it. Do you want to try on the one copy you have? No, you take one half of the mirror and store it somewhere safe, then you can try fixing the other half. If it works, fine, but if it doesn't you haven't made things worse.

With a setup where your data is striped over more than one drive, you can't do that. If that happened, then you'd have to revert to backups, but there are good reasons why that might be the last option.

In those instances I generally resort to making an image before messing with things. Small drive for OS, image if you need to mess with it, generally treat as disposable.

[PlainName]

the great thing about software RAID

It's swings and roundabouts. Software RAID isn't transparent. Hardware RAID doesn't lend itself to SMART monitoring of individual drives. Etc. Depends on what the implementor/user considers important to them.

[Berni]

But any more fancy RAID arrays absolutely requires a working RAID controller to get any useful data from it.

And that's the great thing about software RAID, you're not tied to a specific model and firmware version of controller which may or may not still be available.

Yep for this reason i use unRAID on my homebuilt NAS server.

It provides the redundancy of RAID while even on a failed array i can pull the drives out and read the files from each drive individually on any Linux machine. Tho as a consequence i also loose the speed advantage of using RAID, but i only have a 1Gbit LAN network anyway, so it bottlenecks me to about half the sequential read speed of a single drive. Totally worth the price of unRAID for my home server use case. Pretty much the reason why i didn't go with FreeNAS, since im an electronics engineer, not a IT professional, so i want to avoid the chance of doing something really stupid and corrupting a more fragile RAID array to an unrecoverable state.

[Monkeh]

the great thing about software RAID

It's swings and roundabouts. Software RAID isn't transparent. Hardware RAID doesn't lend itself to SMART monitoring of individual drives. Etc. Depends on what the implementor/user considers important to them.

Software RAID is transparent to everything but the OS, and most hardware controllers do allow SMART access these days. The only real upside I've ever seen to hardware RAID is performance in certain applications, for which consumer drives need not apply anyway.

It is, and this is my opinion you may disagree with if you wish, a needless expense and unnecessary liability in small scale bulk storage.