Come diss my NAS/Server idea!

[PlainName]

In those instances I generally resort to making an image before messing with things.

I run one off every night as a matter of course. I try to have a recent one if I know I'm going to do something dicky, but sometimes these things happen out of the blue, just when you missed last night's job, etc. Don't get fixated on OS stuff - that's just an example off the top of my head. There have been other instances that definitely didn't involve the OS.

Small drive for OS, image if you need to mess with it

Yes, that's what I do. And my small OS drive is 90GB with 15GB free. Thanks Microsoft - even telling Visual Studio to sit on a different drive, it won't fit because the fat-arsed git will plonk 10's of GB in system32 for no good reason.

Sorry, you hit a nerve there

[paulca]

Thanks Microsoft - even telling Visual Studio to sit on a different drive, it won't fit because the fat-arsed git will plonk 10's of GB in system32 for no good reason.

Reminds me of installing SQL Server or Exchange.  "All your computer belong to us now, ok?"

[amspire]

the Netgear Readnas 6 boxes

Have to say that if I were in the market for off-the-shelf NAS, Netgear wouldn't even get on the long list never mind the shortlist. They make some decent non-managed hardware, but anywhere firmware is involved they are terrible. No, terrible it too kind. Atrocious is more appropriate.

Sounds to me like you have never used ReadNAS 6. You cannot compare ReadNAS to, say, modem firmware. It is Debian Linux that has been updated for something like the last 8 years probably at least monthly. I have used at least 8 of the ReadyNAS boxed running 24/7 with never a software issue or hardware failure.

[PlainName]

Sounds to me like you have never used ReadNAS 6

You are correct. My comments are based on many years experience of Netgear products, including NAS kit, but not RN6. Nevertheless, incompetence seems to be baked into the corporate culture there, and unless their brand has been bought to to stick on someone elses OEM badge stuff I see no reason why RN* should be any different.

with never a software issue or hardware failure

When a university had an IP address DDOS'd, the Netgear kit end users didn't notice a thing wrong.

Edit: just stumbled across this:

Neatgear has cocked up its cloud management service, losing data stored locally on ReadyNAS devices' shared folders worldwide – and customers have complained to The Register about only being informed four weeks later.

https://www.theregister.com/2017/04/26/netgear_customer_backups_fried/

[tkamiya]

I've been in this business for long time, as a customer and as a company rep.  Just until recently, I worked for one of the biggest cloud storage/computing supplier.  Proper cloud service of any kind is expensive.  Free service or something given to you as part of hardware purchases aren't.  But they do cut corners everywhere they can, and hide them in fine prints, which you ok'd without reading. 

Personally, not that I have any top-secret content, but I don't trust cheap suppliers ability, budget, or willingness to keep and keep secure my files for free.  I just don't trust them.  My files are valuable to me.  If I have to keep it elsewhere, I will pay premium for it.  Since it's not a possibility, I do it in-house.  (literally....)

Also, I found using old desktop or servers can be a false economy.  Today's machines are quite efficient in terms of power consumption. It's far cheaper to run 2 disks mirrored than 10 disks in raid 6. 

I also try not get attached to company's proprietary anything.  I considered it but abandoned those NAS box ideas early on.  Can I recover data if CPU board dies?  What kind of filesystem are they?  Can "secure protection" be bypassed by me, the owner of the file, to read content on another machine?  All those were considerations.

In the end, I ended up with a new generic Linux server, spare parts, and software raid.  Having worked in hardware/software support for total of 20 some years, I don't want to depend on them to get my precious files back.  I am my own support.

[amspire]

Neatgear has cocked up its cloud management service, losing data stored locally on ReadyNAS devices' shared folders worldwide – and customers have complained to The Register about only being informed four weeks later.

https://www.theregister.com/2017/04/26/netgear_customer_backups_fried/

The last thing I ever wanted to even look at was Netgears cloud service.

It was always a dubious service that was designed to make cloud storage look easy rather then secure.

It was totally optional and unnecessary.

The nice thing about ReadNAS is all the NAS functionality is done using open source tools and you have access to it all.

I don't know of any Netgear family for which you have such complete access to the workings.

It is also pretty nice that rather then have different ReadyNAS's for all the different models, they have one ReadyNAS for all platforms from singe core Marvell processors with 500M RAM to multi-core rack units Intel processors and many gigs or RAM. They just have the same O/S + ReadyNAS available in distro repositories for all different platforms. They are all kept in sync - even the platforms that have been phased out for years.

[madires]

In the end, I ended up with a new generic Linux server, spare parts, and software raid.  Having worked in hardware/software support for total of 20 some years, I don't want to depend on them to get my precious files back.  I am my own support.

A wise conclusion!

[paulca]

I've settled with a 2x2Tb RAID 1 array, called vault.  Backups of bulk storage, like photos/videos/unreplaceable media get's stored or RSync'd here.  My home automation system will run from this drive.  Software "devel" folder also.

OS and /home is on the 256Gb Sandisk SSD.  Rsyncing key config files to vault.

My existing 6Tb "storage" drive will just move over.
My older "bulk" 2Tb storage drive will move over too.
That leaves me a blank 2Tb drive as spare.

I also have another USB enclosure with a 2Tb drive which I use as "offline" backups.  I turn it on, I rsync things to it and I turn it off.  Minimising the drive wear time.

Cloud... is just someone else's computer.
If the service is free.  YOU are the product being sold.

[Mr. Scram]

I've settled with a 2x2Tb RAID 1 array, called vault.  Backups of bulk storage, like photos/videos/unreplaceable media get's stored or RSync'd here.  My home automation system will run from this drive.  Software "devel" folder also.

OS and /home is on the 256Gb Sandisk SSD.  Rsyncing key config files to vault.

My existing 6Tb "storage" drive will just move over.
My older "bulk" 2Tb storage drive will move over too.
That leaves me a blank 2Tb drive as spare.

I also have another USB enclosure with a 2Tb drive which I use as "offline" backups.  I turn it on, I rsync things to it and I turn it off.  Minimising the drive wear time.

Cloud... is just someone else's computer.
If the service is free.  YOU are the product being sold.

2 Tb? That's only like 250 GB. Could've gone for a bigger drive. Even those 750 GB drives aren't huge.

[paulca]

2 Tb? That's only like 250 GB. Could've gone for a bigger drive. Even those 750 GB drives aren't huge.

 

[Mr. Scram]

Putting in extra effort to do it wrong is facepalm worthy indeed. 

[Berni]

You start to take the difference between MB and Mb very seriously once you have dealt with enough memory chip manufacturers.

They never just give you the number of bytes, but always the number of bits cause its the bigger number. So when you are buying a 64Mb flash chip you better be aware this is actually only 8MB

Then again if we are pedantic saying you have 8GB of RAM is technically wrong and should have said 8 GiB (Gibibyte) since RAM uses prefixes of 1024 while calling a 500GB hard drive is correct since they use 1000 prefixes so its actually 465.6 GiB while Windows will then present this as having a 465.6 GB partition. But lets be real nobody uses mibibytes

[Mr. Scram]

You start to take the difference between MB and Mb very seriously once you have dealt with enough memory chip manufacturers.

They never just give you the number of bytes, but always the number of bits cause its the bigger number. So when you are buying a 64Mb flash chip you better be aware this is actually only 8MB

Then again if we are pedantic saying you have 8GB of RAM is technically wrong and should have said 8 GiB (Gibibyte) since RAM uses prefixes of 1024 while calling a 500GB hard drive is correct since they use 1000 prefixes so its actually 465.6 GiB while Windows will then present this as having a 465.6 GB partition. But lets be real nobody uses mibibytes

One is a few percent out and one is an order of magnitude out. The latter is a big difference. Using the incorrect notation seems to be mostly accidental too as typing it wrong isn't less effort, so pointing it out may help. If you order hardware with the wrong prefix there's a good chance you'll get exactly that and there's little you can do at that point. Manufacturers aren't using one or the other consistently either.

Whether not using Gibibytes in incorrect is a bit of a difficult discussion as it was appended after the fact and there's a good argument that GB can mean either. It's good practice to use the ibi notation to avoid the confusion though.

[paulca]

So I got everything up and running.  I had the thing all prepped weeks ago, but was procrastinating and putting off migration, trying to work out if I could do it without downtime.

In the end I said, sod it, downtime will be incentive to get it working.  Took 2 hours.  Biggest delays where messing around with Ubuntu's NetworkManager and finally uninstalling it completely for wasting time.

Performance wise, it's fine.  Disk bandwidth terms...

From internal SSD to USB3 HDD it took 8 minutes to copy 33Gb.

33/8 = 4.1Gb per minute =  68Mbyte per second.  Sounds low to me, but it was writing real work load files, like 10s of thousands of small files and a few large ones.

Network writes from a Windows 10 machine on 1Gbit/s LAN started out around 112Mbyte/s, which is a saturated gigabit link more or less.  When the memory caches filled up that dropped to the 30s and 40s, which again looks pretty low and might need some tuning.  So copying anything less than 4Gb it's flat out, over that it's slow.

Final disk config is TBD, but currently I have:
120Gb SSD OS boot drive Ubuntu headless
240Gb SSD /home [Shared]
2x2Tb RAID 1 "Vault" where current and past projects get rsynced to daily and backed up periodically to external HD kept offline normally.  Stuff I can't just download again goes here.
6Tb bulk storage / media.  [Shared]

Still to allocate:
2Tb brand new
2Tb reclaimed

Probably for the bin, or wiped and sold/donated to good homes.
2x 1Tb SeaGate (6 yo )
2x 500Gb Hitachi (10 yo )

Next project will be to try and change the fan in the SFF PC to a Noctua and maybe consider if the PSU fan can be replaced or if a new PSU is expensive.  Just aiming for silence, at least when the drive bay spins down.

It seems happy anyway.

[nightfire]

Sounds like a nice setup you have done here!

Also, my 0.02 bucks: The HP Microserver is quite popular with lots of "build-your-own-NAS" scenarios.

My own NAS I built years ago consisted of a Silverstone case, where I put originally a 12V Mainboard with the first Intel Atom N270 (notebook low voltage) version in. Booted FreeBSD from a 16GB CF card, had 2x 1TB Seagate Constellation (Video recording HDD) in it, in a mirrored config. Under full load, that rig consumed 24 Watts...
Idle, I came with both HDD suspended to 12W.
Meanwhile, as that board died, an old desktop board with an old i3 Sandy Bridge went in, and boot from SSD- 2 internal HDD, that soon will be pumped up to 3 (have only 4 SATA connectors) and an external USB drive that gets connected from time to time to do an external backup.

In my experience, a NAS with a RAID over several USB devices is doable, but there are so many variables where it may go wrong or cause issues, that I really do not like that.
Internal HDD/SSD are the way to go for me, and some external backup on a USB drive that gets physically disconnected after backup is a way of protecting valuable data.
RAID can sometimes work wonder, but does not protect from fire, overvoltage or a crypto trojan.

[paulca]

an external USB drive that gets connected from time to time to do an external backup.

This is something I do too.  It's much cheaper than using optical or tape.  Keeping it off line means it will last a lot of years.... and you can't accidentally delete or infect files if they aren't connected.

Fine for home use.

For small business use I have provided people with a swappable drive bay that is backed up to every night and they are instructed to swap it with a duplicate in their fire safe every week.