New server is nearly complete

[rrinker]

 I could leave it as-is, but since I already ordered another drive, I'll wait for that before I button it up and put it in the physical spot of the old server so I can get my workbench back.

I have lots of movies, TV shows, music, and pictures. The most valuable data is the pictures, those I can;t just get back by ripping a CD or downloading something. But I'd rather not be bothered by having to fix individual drives and recover/recreate anything, so I've been running a home server for about 12 years now. Original Windows Home Server, and then the 2011 version. The 2011 one was rebuilt twice as I outgrew it, and now it is completely off support (based on Server 2008 R2) and also running out of space.

 Side note - NEVER buy hard drives from Amazon. My server has a failed, nearly new 3TB drive which I ordered from Amazon as they were cheaper than my usual go-to, Newegg. Drive arrived in the static bag in a box with a couple of those air cushions in it - free to flop around and get otherwise well tossed about even if there was no rough handling in shipment. I didn't even bother installing it, I went right to Amazon to return it, specifically mentioning the poor packaging as the reason. All goes well, they next day a new one, with a return label for the original - only it's packed THE EXACT SAME WAY  I was really out of space as one of my ancient older drives had failed so I installed the new one anyway - amazingly, it worked. For about a month. So all drives for the new server were ordered from Newegg, even if it was a couple of dollars more. From Newegg, each drive comes in the static bag, packed inside an ingenious bubble wrap clamshell, then placed inside a cardboard box with additional padding around all sides.

 So, time to build a new server. There's no real replacement for Home Server, leave it to Microsoft to kill off one of the best products they ever made. A headless server with simple to connect clients, no need to build any sort of network other than what you'd already have if you have multiple computers sharing a single internet connection. So after some investigation into what others in the home server community were doing, I decided to just build a Windows 10 machine, run the free versions of Veeam backup to backup my workstations, and continue to use the same drive pooling software, CoveCube's DrivePool. Found a motherboard with enough SATA ports, and on top of it all, my first AMD CPU since the Athlon XP days. Running a Ryzen 3700 in this one, 6c/12t. I put 32GB RAM in it, so I can also use it to run VMs for testing and whatever, mostly work related stuff. Setting up the shares is not quite as easy as with WHS, remember I am a networking/server guy so I can do all this, but no one else in the house is any sort of computer expert and this has to be easy for them.
 But, it's now all working, got all the files copied from the old server, the old server is removed from the cloud backup (I use Crashplan) and the new server is being backed up. Everything but my workbench computer is set up - the server is currently sitting on my bench using the monitor and network connection of the workbench computer until I put it back in place on the floor where the old server was. I started with 3x 6TB drives, I have another on order, and I stuck in one of the good 3TB drives from the old server (I had 3 of them, 2 good ones plus the one that dies, easy enough to track because the drive pool software reports the serial numbers of the drives). As it sits right now, the pool size is 19TB, with 5.8TB free (it does folder duplication to store each file on 2 or more drives - so my actual data size is half the used space - I could also have it duplicate REALLY important stuff to 3 drives - which I might do for the photos). The incoming 6TB drive will add another 5.8TB of space to the pool, so I will have a server holding nearly 25TB - I still remember my one job where we ran 6 IBM 3090 CPU complexes, we had a whole ROOM twice the size of my basement that housed a whopping 13TB of storage. Plus there's still one more drive bay in the case (and with 3.5" to 5.25" adapter rails - another 3 bays to use) and one more SATA port (after I put the 6TB drive in I'm waiting for). So I can either use the other good 3TB drive from the old server, or get another new larger drive. I can always go back and upgrade one drive at a time to even larger ones if I ever need to. Oh, and there are still 2 trays to mount 2.5" SSDs - I WAS going to put in a pair of 2TB drives to use as a fast cache, the drive pooling software supports doing this. If prices keep dropping, I do want to upgrade my network so at least the server is on a 2.5Gb or 10Gb connection, then maybe the fast write caching could come in handy.
 I also have Plex on here to feed my TV.

[gmb42]

Not trying to be a PITA, but have you read para 2, c clause (v) of the Windows 10 licence:

2.   Installation and Use Rights.
...
c.   Restrictions. The device manufacturer or installer and Microsoft reserve all rights (such as rights under intellectual property laws) not expressly granted in this agreement. For example, this license does not give you any right to, and you may not:
...
(v)   use the software as server software, for commercial hosting, make the software available for simultaneous use by multiple users over a network, install the software on a server and allow users to access it remotely, or install the software on a device for use only by remote users;

It's always been unclear to me if this restriction applies to such things as SMB shares, which are definitely provided by the OS, or applications such as Plex.

[Monkeh]

I doubt he cares for home use..

Might care when it decides the machine is not in use and it's fine to reboot for updates in the middle of a film.

[madires]

If you like to go Open Source have a look at FreeNAS.

[rrinker]

 If they truly enforced that, there are thousands if not millions of places using it for remote access. Microsoft's OWN server system allows access to client desktops via the server, breaking the whole "remote access" thing. I'm not running a web server on it, and it's not accessible outside my home network.

 And you're right, frankly I don't care. If you are not allowed to share files from a Windows 10 machine, then why does it even have a file sharing feature?

That may also be version dependent - I wouldn't touch Windows 10 Home with a 10 foot pole.

I looked at FreeNAS. It has some neat things, like the Docker containers, and tons of pre-built ones. However, I still can't get past the way they do the drive pooling. Just not for me.

[Monkeh]

However, I still can't get past the way they do the drive pooling.

Huh? What's wrong with it?

[rrinker]

 Main thing is the way it uses a parity drive and the size of that drive controls the total pool size. Drive Pool (not a Microsoft thing, a third party product for Windows) uses completely native NTFS features to make sure your files are stored on 2 or more physical disks. Each disk can be any size, mix and match at will, can be emptied for replacement, can be read by anything that can read NTFS, and is size limited only by a Windows volume size limit (good luck getting close to 4 Exabytes even with the biggest drives available today).
 It's been around for some 9 years now, and is highly mature. And it only uses native elements of NTFS that have been in there at least twice as long. There are multiple options for prioritizing how files are placed (keep all drives equally full, prefer specific drives, etc) and they expose the API for additional plugins - one they have which I was originally going to use but changed my mind allows for SSD cache where it preferably writes new files to the designated drives and then moves the files off those drives as a priority. They also have a complimentary product that allows you to seamlessly add a cloud-based storage to the pool - so your pool can contain local physical disk as well as any of the well known cloud-based solutions like AWS, Azure, Box, etc.

[Monkeh]

Main thing is the way it uses a parity drive and the size of that drive controls the total pool size.

So.. it's RAID, and you use multiple vdevs if you want to mix and match. But okay. Hit-or-miss file location across a randomised NTFS mirror sounds.. less enjoyable to me.

[rrinker]

 The whole point is NOT having RAID, because that limits drive sizes and increase vulnerability when rebuilding. ANd is not easy for regular people, not that this is applicable here.
The thing is, the duplication is invisible. I see a drive. With folders and files. I don't know what the replication limits happen to be, but it's at LEAST up to 3 copies, and I think you can do more, although having more than 2 copies in the same physical box is diminishing returns on any actual redundancy. But it's configurable, no duplication, 2 copies, 3 copies, etc. But other than setting this when creating a folder, you do nothing. And there is little need to know which drive a given file is physically on, when it's on more than one of them. If a drive fails, any file now showing 1 replica is already duplicated to another drive, it doesn't sit there in a degraded state until you intervene and replace the failed drive. And if you want to manually remove a drive to replace it with a larger one or whatever, it will take care of that for you as well.

The same style of disk replication was in the original Windows Home Server, I've been using that for 12 years. Despite drive failures, I've never lost any data. And when I built the second server, I transferred all the data over simply by attaching each drive from the old one in turn and copying the files. No custom format with limited readability.

I will add that in all my years of using Windows NT and descendants, with NTFS formatted disks, I have NEVER lost data unless the drive had a physical fault - controller error, stepper failed, spindle motor failed, etc. Corruption caused by loss of power? Every time that has happened, I have been able to take the problem drive and put it in another computer, or use some sort of boot disk with utilities, to run CHKDSK which fixes up the index pointers and then the drive. It is an extremely robust file system. 

[Monkeh]

I was doing a whole quoted thing but..

Basically, you've described something different in implementation but effectively identical in behaviour and constraints to any other mirrored RAID layer or filesystem. Nothing wrong with that, but.. it's the same thing with a different window in front of it.

Seems fine. But the whole idea of relying on Windows, especially 10, I can't get past.

[madires]

Yep, Windows' Storage Spaces is a flexible RAID. See https://support.microsoft.com/en-us/help/12438/windows-10-storage-spaces for a brief overview. If you set up a two-way mirror space with a 2TB and an 1TB disk the virtual drive is still limited to 1TB, because you can't mirror 2TB data on an 1TB disk. Under linux you would run a LVM RAID to support adding disks and growing volumes. If you think that having all disks in one box is a SPOF you might consider to deploy something like Gluster. BTW, there's another way to duplicate data across multiple disks without any RAID technology: rsync. Add a time schedule for different disks and you'll have also backups.

[rrinker]

 It's not Storage Spaces. Storage Spaces is what they were trying to get to work for WHS 2011 but failed miserably, so they released it with absolutely no data duplication system, which is where companies like StableBits with Drive Pool stepped in - the previous version of WHS there was this same sort of file level replication. People were made, but also happy, since Storage Spaces uses a different drive format and you can;t just pop the drive in any other computer and read the data, there is a recovery process. In large environments, we don't bother with Storage Spaces, the server, usually a VM, is on some form of RAID as implemented by whatever SAN they are using, and if they want copies in another place, DFS is configured.

It's most like RAID 1, but at the file level, not at the hardware level, so if the drive sizes don't match, there is no unused space. And it doesn't interfere with things like TRIM when using SSDs.

As to the other comments, you Linux guys make me laugh. I have Linux in a VM, I also used to have a Linux box running to control my model railroad. But going back to my earliest Windows installs (I skipped right over 286 machines and went from an 8MHz XT clone to a 386, so Windows 3.1, and also dual booted Linux 0.89a on that one) I have NEVER had any problems with reliability. The only failures have been hardware faults crashing the box, and if the power supply dies, that will kill a Linux machine just as dead as a Windows machine. I've been using Windows 10 on all of my systems since they first offered it, and the things I see posted here - I really wish I could see what people are doing to break their computer so bad. This machine I am typing this on right now is 9 years old (I have upgraded the SSD and the video card over time - but this is the next one to go) and was Windows 7 until upgrading to 10 - and I have NEVER had to reformat and reinstall Windows because my computer "got slow". Still chugs along, runs what I need, but playing with the new server has now spoiled me, with the faster processor and memory, and a PCI M.2 SSD, it's just way faster.

Even the old server is still good, I probably could have just put some larger drives in it, but as it is based on a 12 year old operating system (and the hardware is just as old)  and support for it is fading fast - the provider I use for offsite backups no longer supports it, although it was still performing backups - persuaded me to build new. I've been thinking about it since last year, after the fiasco with Amazon purchased hard drives. On one of my business trips last year, one of the IT guys told me about FreeNAS, so I spent a lot of time researching it because it sounded pretty good at first. But there seem to be some questions about the other things I do with the box besides be a big file share - old or outdated info on backups, like no one has implemented, or at least talked about how they implemented, newer versions of both client side backups and server backup. And I was really wanting to go with SSD fast cache for the disk, which it seems FreeNAS does not do, although you can make it fast by making the parity drive an SSD - which is the other issue.

All in all, I don't expect this to be any less reliable than any of my preceding machine. Which is to say, totally reliable. It's one less chunk of spinning rust than the odl one, but also a different brand, so we shall see.

[madires]

I've only tried to offer alternatives without licensing nightmares.

[Monkeh]

It's most like RAID 1, but at the file level, not at the hardware level, so if the drive sizes don't match, there is no unused space. And it doesn't interfere with things like TRIM when using SSDs.

If you don't have at least two drives of the same size, you can't fully duplicate, so drive size is a moot argument. If you arrange a ZFS pool with one zdev consisting of 2x3TB mirrored drives and a second of 2x6TB mirrored drives, you have 9TB of space and none (beyond normal overheads) wasted. This is no different to having 2x3TB and 2x6TB drives with file-level duplication.. I can arrange the same strategy with LVM or MD, too. Now, I guess you can't select bits and pieces of that to be non-duplicated, but if you want Schrödinger's data, just use one drive to begin with.

As for TRIM, I would have to check into capabilities with ZFS or LVM. MD, however, does not interfere with TRIM for mirrors.

As to the other comments, you Linux guys make me laugh. ... The only failures have been hardware faults crashing the box, and if the power supply dies, that will kill a Linux machine just as dead as a Windows machine.

I've had both go through power failures and so far Windows machines have needed a lot more hand holding to recover, if recovery is even possible. It effectively wasn't for a recent customer who had a partial drive failure on a laptop during an update - the only viable recovery method was an install-in-place of Windows, something new to 10 I was highly wary of. It worked and no harm done, so hats off to Microsoft - they invented a workaround for their unrecoverable nightmare. Other systems have been easily recovered from such incidents since, uh.. always, really. Identify bad file, replace bad file.

Generally, however, a mere power failure doesn't cause a problem to either - it's the 21st century, we have journalling and transactional filesystems. Some of them even have advanced data integrity features (NTFS need not apply - it's old, but not bad). Sidenote: Nobody even mentioned Linux until now. FreeNAS ain't Linux.

I really wish I could see what people are doing to break their computer so bad.

Using them. I have fond memories of DRM software causing CPU resets immediately after login with no evidence of the cause. WHQL approved driver!

On one of my business trips last year, one of the IT guys told me about FreeNAS, so I spent a lot of time researching it because it sounded pretty good at first. But there seem to be some questions about the other things I do with the box besides be a big file share - old or outdated info on backups, like no one has implemented, or at least talked about how they implemented, newer versions of both client side backups and server backup. And I was really wanting to go with SSD fast cache for the disk, which it seems FreeNAS does not do, although you can make it fast by making the parity drive an SSD - which is the other issue.

FreeNAS both offers multi-level write caching (because ZFS does), multi-level read caching (because ZFS does), and parity-free duplication (because ZFS does). As for backups, backup strategy is a whole long, complicated subject and will depend on the platform you're backing up. Return on your time investment seems poor.

I'll also address a previous pont: Your duplication scheme is no less vulnerable than traditional RAID (or LVM, or ZFS, or or or..) in event of failure and restoration of duplication or parity. How could it be?

[rrinker]

 You're right of course, if only 2 drives were involved. I have 5. ANd one is half the size of the other 4., a leftover from the old server. I have one more usable leftover I could fit in there, but I'll just keep that one as a spare for now, I already have twice the space I had on the old server even with turning on duplication for things I don't really need to duplicate - and this is where it comes in handy. I have 2, maybe 3 classes of data stored on my server - top priority is the "no possibly way to replace this" stuff - mostly photos. Since I have the space, I might enable 3 way duplication instead of 2, plus it is backed up off site. That's actually one of my smaller data sets - I don't have a DSLR shooting monster file size RAW images. Next is the "I could recreate all this but it would take a long time and a lot of effort" which is mostly my music files. I have all the CDs stored away and could go through and rip them all again if it all disappeared, but I'd rather not have to do that. So that is duplicated and backed up offsite. Finally, there is the "it's stored here to be easily accessible from anywhere on the network" not critical stuff that if it suddenly disappeared, I can recreate from any number of sources, including just streaming. Mostly TV and movies. On the old server, I didn't even have this duplicated. I do now, and still have more free space than I had total space on the old one.

 Where I stand right now, I have over 12TB free. If one of my 6TB drive fails, I still have, worst case, another 6TB free - the Drive Pool software will automatically add a duplicate for every file that has lost its duplicate due to the drive failure. Like having a hot spare. ANd it will shift data around to make space if it has to, since I'm not using the setting to fill the first disk 100% before going on to the next disk. Like having a hot spare but it's not reserved as only a hot spare like a traditional RAID. Since I added the 4th 6TB drive and the 3TB drive AFTER I had copied the files from the old server, those two drives are mostly empty right now, since the balance option to even out the free space is very low priority right now.

 One thing on the new server that will use more disk space than before are my workstation backups. Free Veeam doesn't do dedupe like the built in backup from WHS, so maintaining a similar retention cycle will use a lot more disk space. On WHS, you couldn't place the backup on a duplicated volume, not  problem using Veeam for backups, so the backups are replicated.

 I've had Windows machines lose power during updates - they've always recovered. Worst one sat there for about an hour, I was preparing to have to rebuild, but it figured itself out. There is a log generated for this, and if you boot to command prompt you can just "fix the file" that it got stuck on. No modern machine is going to be as completely flexible as my old TRS-80 though - don;t need a certain DOS feature? Just delete the file and save some floppy disk space. Didn;t break anything, just meant that feature couldn't be used. In the days of 180K floppies, that was important. No one cares now that you can throw 64GB RAM and a 14TB hard drive in a machine.

 I guess I don;t use exotic enough hardware - though my two different model railroad interfaces have to be fairly unique. Only other things I really use besides fairly standard keyboards and mice, and printer, my scope has a USB interface, and I have one of those USB logic analyzers. Outside of PIC and Arduino programmers, that's about it. I've had devices that just plain didn't work, but never had a drive that totally blue up Windows. Not in recent memory, not on my own machines or at any clients.

 My work laptop came joined to the company domain, which I don't do, and a bunch of unnecessary stuff installed, so I used the Windows 10 factory reset on it, works very well. Came up like it was a fresh install of Windows 10, without having to reinstall myself (not that it take long from a flash drive, Windows 10, Server 2016, or Server 2019 install in minutes now). The one thing I always see are people running Windows 7 who are forever reinstalling the OS, every 6 months or more. That is something I have never had to do, nor do I see any of our clients doing. When I was regularly playing different games, I was constantly installing an uninstalling software, still didn't have to do that. Slightly more recently, I tried about 3 different EDA programs installed and uninstalled each one on this Windows 10 machine - still don't see a need to restore the system to defaults.

 The server, I've even blown away a whole bunch of Windows 10 junk that certainly isn't needed on a 'server' like the Xbox stuff, Mail, Maps, Weather, etc. Yes, you CAN granularly uninstall a whole bunch of that consumer crap. Not that the disk space is critical, I just don't need all that stuff running.

[Monkeh]

You're right of course, if only 2 drives were involved. I have 5.

I mean, my example was for 4, and it works just as well for any number. Performance drops as you increase the complexity of the mix-and-match mess, but..

I've had Windows machines lose power during updates - they've always recovered. Worst one sat there for about an hour, I was preparing to have to rebuild, but it figured itself out. There is a log generated for this, and if you boot to command prompt you can just "fix the file" that it got stuck on.

I really wish that were the case. If it didn't have customer data in it I'd send you the image to try out that theory.

[rdl]

I think with Windows the main issue with bloat is the registry, nothing is ever removed completely when it's "uninstalled". Windows has to read all that old crap frequently. Modern hardware and upgrades tent to minimize/mitigate the problem.

As far as the server goes, no way I would hold on to all those old, mismatched drives. When I built my FreeNAS system I started with all new drives sized to fit my current needs, plus about 50% for future growth.

At any rate, whatever works for you and you are satisfied with, is what you're going to use.

By the way, how does Drive Pool deal with bit rot? Does it do any kind of periodic integrity checking? What about snapshots?

[rrinker]

 That's why I only used one drive from the old server. Though through various updates I made on that thing, all the old drives were 2TB or larger. It originally started with 750GB drives, then 1TB drives got added, then 2TB, and finally 3TB.  Things change    When i started out, 3x 750GB seemed like a ton of storage and I'd never fill it all. Well, since the old one was filled, nearly so, anyway, there was about 1.5TB left, I built this new one with double the storage. HOPEFULLY this is enough for a while, but that's the thing - 2 years from now, if I start running out of space, I can swap a couple of the drives for whatever the best price point drive is then. Instead of having to buy 4 or 5 new drives all at once.

 It would be a waste to just store or worse, destroy and recycle the otherwise good 3TB drives from the old server, so with a free bay and the right cables, I stuck one in the sew server. it's not insignificant space, and it's already been paid for, so why not use it? The old 2TB drives are destined to live in a drawer, I think. Being on 24/7, these drives all have very low power cycles and very low head cycle counts. Since the old server was also well cooled with dedicated fans over the drive bays, they didn't run hot. So there should be lots of life left in them. The 3TB drives still have half their 5 year warranty left. So why NOT use them?

 StableBits also has a complementary product called Scanner which periodically does surface scans and also monitors SMART data. If any drive is in danger of failing, it controls the pool to remove data and migrate to other disks, and will remove the failed drive. It will also do things like throttle disk IO if a drive exceeds a set temperature threshold. It also sends alerts via email, SMS, Twitter, Speech, or mobile device pushes.

 

[Monkeh]

HOPEFULLY this is enough for a while, but that's the thing - 2 years from now, if I start running out of space, I can swap a couple of the drives for whatever the best price point drive is then. Instead of having to buy 4 or 5 new drives all at once.

.. and, again, this is nothing FreeNAS or pretty much any other system can't do.

I'm not trying to convert you here, just make you aware that there's nothing earth shatteringly unique in the product you're using.

What about snapshots?

Volume Shadow Copy. NTFS is a moderately advanced filesystem - COW and snapshot functionality is there. Only arrived in real force with Server 2003/Vista, with some ugly backwards compatibility problems.

[rrinker]

 Never said it was unique, although before Microsoft discontinued the actual WHS product, it was pretty unique - sure you could get other systems and make or find add ins to gain all the other features like media sharing on top of the storage, and maybe even a backup that could dedupe like the WHS backup (at the time, the only place I saw dedupe like that was in commercial backup products, usually as an expensive add on even then).

 I actually remember looking at FreeNAS in the early days. I had a spare machine (which could run Linux better than my then current machine - something that soured me on AMD and ATI graphics for a long time - this new server is the first AMD CPU machine I built in 15 years) and I saw mention of the storage system that could run from a USB stick. I had lots of old drives which, taken individually, weren't much, but fully loading that old machine, I could have a pretty decent amount of storage. No need for media servers back then. But it couldn't do data protection unless I built a RAID of identical disks, which I didn't have, so I filed it away as neat idea if I ever buy s stack of identical drives.

 Seriously - most of this stuff is my day job. I don't want to do it any more when I get home. When I was younger, I ran all sorts of servers at home. Mostly cobbled together from leftover old equipment, but they were reliable. In the Server 2003 days, I had 3 servers plus a software firewall running in my house. Apart from water pouring in a basement window right into the crt monitor and a failed fan here or there, this cast off hardware served me well - the servers were all Pentium 2 machines, the firewall was a Pentium 233. I had all sorts of gadgets - a NEC 6 disk CD changer on the file server, even a DAT backup for the whole thing. Eventually as the hardware really aged I just got tired of dealing with it all day at work and then more when I got home and I ended up scrapping it all. For a few years I just relied on what was in my local desktop, until finally moving to the HD TV era and discovering WHS and media players, my first one was the original Popcorn Hour player.