Please recommend me a NAS with minimum 6x2.5" bays and no FAN

[David Hess]

I have seen the used PERC ones which came out of data centers before.

I found out Backblaze uses cheap 2 and 4 port SI controller boards with SI port multipliers.  They report problems with non-SI controllers and port multipliers so avoid using the Intel (and AMD?) SATA ports on the motherboard.

[DC1MC]

I've won these two Shuttle XPC on  >:Dbay, for less than 80EUR together, one silver and one black, let's see who radiates better  .
Also I've got myself some kind of Chinesium Powermeter to see how low can I go ), I'll have it checked with the HW guys in the office to see if it's totally bogus or not.

The silver XPC is a Dual Core E6500 - 2,93 GHz w/ 4 GB RAM, the black one is a Dual Core E5400 2,7GHz  w/ 3GB Ram SG41J1.
I'll see when they arrive what kind of PCI slot they have, to get the SATA 8 port adapter and cables. In the mean time I'll try to see if I can quiet the PS and the CPU cooler fan.

Also two "defective" 128GB SanDisk SSDs, for 9,95EUR a piece, shipping included, to use them as boot disks, there is a guy that sells them on DE bay as "defective, cause unknown", he sold hundreds and one buyer said: "Wrong description, I've bought 10 of them and none is defect..."   

Enjoy the pictures and cheers,
DC1MC

[mariush]

Those meters are reasonably accurate... should give you actual power +/- a few watts.
Also, usually these report correct values from around 3w and up.... don't expect to measure stand-by power with pc shut down (~1w)

[Hemi345]

I bought two over those XPC "glamour" Shuttle cases and the power supplies are complete crap.  They were anything but quiet since they had 3 fans spinning inside of them and the aluminum chassis amplifying the fan noise. They both ran 24/7 in my office (clean enviroment) and both PSU's died after about 1.5 years. The genuine Shuttle replacement PSU died after about a year.  I frankenstein'd a standard ATX PSU on the other one by just sitting it on top and running the power cables through the hole where the old PSU used to be. It ran that way for a few years till the fan on the chipset died a few years later. 

A steel mini tower or HTPC case that can accept one or two 120mm fans is your best bet for silent operation with a little performance mixed in and still look nice.  In my HTPC case, Silverstone, like this one

 I installed an Athlon 3ghz dual core with 8GB ram, an SSD for OS and apps, and 1TB WD Black for recordings. The HDD is suspended using bungees in one of the 5.25 bays for near silent operation.  The CPU has an old Arctic Cooling heatsink on it with the fan removed, this one:

PSU is a 500W Corsair model that is dead silent.  I've never seen the fan run on it, maybe it's broken haha.  One single 120mm fan wired for 7V in the case cools the entire thing and has run great like this for the last 7 years 24/7 decoding HD streams and comm skipping recordings. I cannot hear it more than a couple feet away in a dead quiet room.  Since we use it as a front end for our TV, it needed to be silent.  The great part is I have plenty of room for more HDDs, fullheight PCI/PCIe cards, etc and it looks like another piece of audio gear sitting next to the HTPC receiver.

[DC1MC]

Just a quick update:
- The silver XPC arrived, the black one will be here today.
- The power meter arrived as well, the GLS driver of course just left a card   
- I have happily destroyed the two SanDisk and a OCZ of 120GB each, doing thermal testing, around 85C they stop, around 95 they die if under power, one of them with magic soul smoke even. Pictures will follow 

I will start "silencing" testing, Hemi345 is right, as far as I remember, the Shuttles have a large CPU cooler fan that is easy to control a whiny 5cm power supply fan that whines like one of these hysterical little dogs. I will start to get rid of it first.

Question:

What should I use for the system disk: WD Green, Kingston or Crucial, all 240GB, any suggestions backed by actual experiences are appreciated.

 Cheers,
 DC1MC   

[Jeroen3]

- I have happily destroyed the two SanDisk and a OCZ of 120GB each, doing thermal testing, around 85C they stop, around 95 they die if under power, one of them with magic soul smoke even. Pictures will follow 

Flash memory definitely cannot handle high temperatures. At 85 degree ambient the retention of SLC memory inside microcontrollers drops to a few years. And that's large flash.
SSD's have very modern multilayer tiny flash.

I would recommend a Samsung drive. I've used Crucials, and they fell short. They couldn't handle combined IO (sequential and random) very well. The Samsung I have now can do this a lot better.
Crucial used to be the cheapest, yet the competition in price has almost died. You have to look at the controller and firmware today. The availability of backup capacitors, and cache memory.

Edit: Also, make sure you use the correct filesystem, and no hardware raid.
https://blogs.technet.microsoft.com/filecab/2016/11/18/dont-do-it-consumer-ssd/

[DC1MC]

Uhmm, the boot/system disk in my NAS situation will do almost nothing from disk I/O point of view, it will boot the system and hold some logs, what needs to have is a bit of reliability.

Let's see what controllers do those babies have:


WD Green                         Silicon Motion SM2258XT
Kingston SSDNow UV400   Marvell 88SS1074
Crucial BX500                   Silicon Motion SM2258XT

Now what ?

[ogden]

Edit: Also, make sure you use the correct filesystem, and no hardware raid.

No hardware raid? - Could you please provide specifics?

[Hemi345]

Going by Amazon reviews, the Samsung 8xx series seem pretty good, I've no complaints on the one that I have in a laptop.  I've had great luck with Corsair Force and Sandisk Ultra and Extreme series in both home and work computers.  The Corsair models have been running 24/7 in workstations for the last 6 or 7 years.  The one in my HTPC is an OCZ Vertex 60GB that is almost 10 years old.  I honestly think that if you don't cheap out and keep temperatures under control, most will last for a really long time, especially if there are very little writes. 

[DC1MC]

=== START , YOU COULD SKIP TO NEXT === for some mildly interesting technical stuff !!!
Well, to start I seem to be able to get the purpose of my project here, that is a cheap, silent, but reliable high capacity NAS, for the cheap and reliable I'm willing to sacrifice the speed and the multimedia capabilities.
So I'm not making a real-time 3D 4K/120fps video encoding system for a high-end porn studio  , neither some kind of fast office/desktop publishing machine, also not a gaming gear.
It's a NAS, speaking of which, now, in this moment, there are established companies selling frigging single ARM core boxes for close to 1000EUR as high performance NAS!!!
And you gentlebeings are proposing me Ryzens, modern high-performance (and high-consumption) motherboards and top shelf (and price) Samsung EVO with 500MB/s speed for the frigging boot disk  !!!
What should I do with this monsters ?!?! The connection with the house router will be via 1Gb network interface (the router has only one available) and the access will be mostly via wireless with 3-4 devices accessing it simultaneously. If all this will get over 100MB/s I'll be amazed. Maybe one iSCSI mount with my workstation laptop that is wired connected and that's it. The data disks in the RAID will need to be a bit less crappy, but not the boot disk (btw. I've started the silver box, with an Intenso (crap de la crap) USB stick, goes in 21s to the graphical desktop of Linux Mint 18)
=== END 


Good, now let's go to some interesting stuff before is too late, so I've heated and destroyed two SandDisk SSD Plus 120GB and one really old OCZ Agility 2 120GB.
Contrary to popular opinion in ALL THREE cases what failed was the controller and NEVER the flash. Doing dd if=/dev/urandom of=/dev/sdc bs=100M count=100 for random data and then doing SHA256 until the last moment ALWAYS produced the same hash value.
Then some DISK NOT READY and whatever other controller related failures appeared, then the disks disappeared for ever, that is, the disk's controller became inert and was never recognized again. One of the controllers even failed spectacularly spitting out smoke on the second cycle of heating long before reaching 90C, around 70-75.
So this kind of settles the mystery of failing disks that are not seen at all by the bios, the controller itself dies and it seem that the most sensitive part is the SATA differential transceiver block. The OCZ disk has TSOP flash and I could bet that if I unsolder it and test it it will still be a good flash. Of course that in the long run the storage quality of the flash degrades with high temperatures, but the controller will be long dead before this happens.

The second big discovery: these cheap consumer SSDs are DESIGNED to fail, please look at the picture of the PCBs, the small ones are the SanDisks and the big one is the OCZ (at its time was really friggin expensive), these guys are small and thin, the cases are HIGHLY ISOLATED, hermetically sealed plastic cases where none of the chips were touching the case body and they were kept at a distance by thin bumps. Of course exactly ZERO air circulation as well inside the case.
The OCZ has 1/2 of the case of metal but no thermal contact with the chips and the metal is ON THE OTHER SIDE of the controller so I think that a huge improvement could be made by just discarding the cases and thermally bound the boards on some metal surfaces to serve as proper radiators.

Have a look at the boards picture and let me know your opinion, now is too late, but tomorrow I can picture the plastic shells as well (and a thick plastic it is, was a pain to open it).

 Cheers,
 DC1MC

[Jeroen3]

Edit: Also, make sure you use the correct filesystem, and no hardware raid.

No hardware raid? - Could you please provide specifics?

The hardware controller must supports SSD's (eg: trim/align support), and the SSD must support being in a RAID configuration. With above mentioned consumer grade disks, I do not see this happening.
Software raid will be the most suitable situation for this, since this provides the most support for all sorts of disks. (most consumer grade nas boxes use software raid)

[DC1MC]

Edit: Also, make sure you use the correct filesystem, and no hardware raid.

No hardware raid? - Could you please provide specifics?

The hardware controller must supports SSD's (eg: trim/align support), and the SSD must support being in a RAID configuration. With above mentioned consumer grade disks, I do not see this happening.
Software raid will be the most suitable situation for this, since this provides the most support for all sorts of disks. (most consumer grade nas boxes use software raid)

This is something that I agree with 100%.

 DC1MC

[DC1MC]

Eh well, today I've encountered again the typical ebay seller imbecility, the black XPC came miserably packaged  in an improvised box made by some cartons taped together and 1 (one !!!) foil of the weakest bubble wrap I've ever seen in Germany, and I've really seen shitty bubble wrap. Of course the DHL took a perverse pleasure of kicking it around until they totally destroyed the front panel, breaking all the stands 
Don't know what else is broken, I've open it and as opposed with the silver XPS, this one is full of dust, really prolly the parents bought it for him, this really pisses me off, because he took the time to dismount the HDDs and whatever crappy DVD it had, but he couldn't be arsed to do a minimal cleaning. 
So enjoy the EBAY complain picture of the poor broken XPS .

By the way, both NAS wannabies are Shuttle XPS SG41J1: http://global.shuttle.com/news/productsSpec?productId=1382

 On the other side I've got two of these babies Adaptec ASR-5805 512MB 8-Port PCI-e for 17.50EUR a piece, shipping included, cables included :



https://www.adaptec.com/nr/pdfs/ds_series5.pdf

Hopefully they will talk with my consumer SSDs, as far as I can see, they are fully supported by Linux, yes I know that they are 3Gb/s and no, I don't care, actually I prefer them like this, less stress to the SSD controller transceiver.

Also the Chinesium mains wattmeter has arrived as well (in one piece this time) and seem to work, I'll tell you good after comparing it with a professional wattmeter.

And I'm still waiting for reasonable suggestions for the boot disks that are not Samsung EVOs 

 Cheers,
 DC1MC

[ogden]

On the other side I've got two of these babies Adaptec ASR-5805 512MB 8-Port PCI-e for 17.50EUR a piece, shipping included, cables included :

Good find considering price    3Gb/s for (1Gb) networked *storage* server is more than enough.

And I'm still waiting for reasonable suggestions for the boot disks that are not Samsung EVOs 

Take Intel then Don't do thermal testing anymore. Baking MLC SSD at 95oC is not testing but scrapping. Even 85oC most likely is deadly, exceeds max temp specs for most SSDs and electronics as such. Limits are set for a reason.

[Halcyon]

what it needs is a half height PCI card with 6 ports

Good luck finding that.  If I knew of one, I would be using it.  Reasonably priced PCI and PCIe SATA adapters are rare enough that it is often much more economical to buy a "gamer" motherboard and disable most of it just for the extra SATA ports.

8-port SAS/SATA controllers are very common (and cheap to buy second hand). You can use SATA drives with a SAS controller, but not the other way around. You just need yourself a SAS to SATA cable. I use LSI controllers in my NAS machines and they work a treat. Some models can be flashed into IT mode (using firmware provided by the manufacturer) so you can use them with ZFS. My second NAS runs FreeNAS in a virtual machine (under ESXi) and the SAS card is passed directly through to the guest OS so that it has direct control of the hardware. It works very well. If you plan to do this, you want to find a controller that uses the LSI SAS2008 chipset.

[DC1MC]

@ogden said: "...Baking MLC SSD at 95°C is not testing, but scrapping."
^^^^
This, exactly this, is why I've backed them, I've heard so many stories about these "horrible MLC" SSDs, that die and suddenly disappear from the BIOS, never, ever, to be seen again and "the data is irrecoverably gone, dammed be the MLC flash...".

And I've actually empirically proved that what actually dies it's the SSD controller, it has nothing to do with the flash (didn't backed it directly at 95, I've done long other reliablility tests with lower temps, never to have a data integrity problem) , and the plastic clam shells are SPECIFICALLY designed to hold the temperature high (I'll post some pictures), so it's kind of planned obsolescence, I could bet that the high-end drives have metal parts in the case, even full metal case AND good thermal contact with the chips, and this keeps the disk alive more than some magic flash or controller sauce.

I'm not talking here of really long and intensive R/W operations, but this should be dealed of by the "smart" controllers that should let one know about these issues and even in the worst case, one should get data integrity errors (hopefully signaled in advance by the RAID algorithms), not disks that suddenly pop out of existence taking ALL the data with them.

If I'll ever get a defective high-end SSD, I'll perform a dissection to confirm my theory.

This changes a bit my NAS plans, because now I'm planning to eject the electronics from the shell (warranty be damned) and use a proper heat spreading and cooling technique, instead of putting them in those cramped (also plastic) 6-bay enclosures.

 Cheers,
 DC1MC

 

 

[ogden]

And I've actually empirically proved that what actually dies it's the SSD controller

I am afraid that you did not prove anything. To prove that it is controller that died and only - you have to replace dead controller chip with working and run full diagnostics of SSD proving that flash chips are intact and controller is only part that failed. Did you do that? What if flash died and controller is fine, just unable to read it's firmware (from flash)?

[mariush]

There's no point.

Writing data on the SSDs is what consumes power, and you'll only write for brief periods of time. Natural air cooling will take care of moving the generated heat.
When you're reading data from SSDs you're basically only dealing with the idle power consumed by the controller, which is what... 0.3w to 1w ... that's nothing.

I wanted to write something about the previous post but I was too lazy

Drives like those Sandisk 120GB ones have a pretty clear warranty : 5 years or 80-100 TBW (terrabytes written) over the warranty of the drive. Some go even further and specify "as long as maximum 0.8-1 drive writes per day" or something to that effect.
Basically, 5 years is so chosen that if you divide that 80-100 TB figure by the number of days, you get an average writes per day that's less than a full drive.. ex 5 x 365 = 1825 days.. 100 TB / 1825 = ~ 55 GB per day.

Keep in mind that you're dealing with 120 GB drives... if you assume an average 200 MB/s of write speed, you're writing 1 GB in 5 seconds, or around 10-12 GB per minute. A full drive would take 10-15 minutes to fill up.
There's some amount of air inside the case of the SSD and there's some inertia...even if the controller and the flash chips warm up with the writes, it takes SOME time for the temperature inside the SSD to go up by any significant amount
If you have a SSD staying on average at 30-40c, you can afford to see it jump up to 50-60c when you're done filling it with data... it will gradually cool down. If it's screwed to a case, just the cooler metal will suck heat away, the plastic will also slowly cool down and so on.

Yes, they cut some corners with plastic cases on SSDs but that's basically price cuts to keep SSDs competitive.. metal cases are more expensive. BUT, for the way they're meant to be used (a few GB or tens of GB of writes per day, it's a perfectly adequate product, the chips are kept at reasonable temperatures).

If you do decide to open them up and add heatsinks, imho it would be enough to put just one tiny heatsink on the controller chip, one of those heatsinsk you see used on ram chips.
You'd do much better cooling just by having a 92-120mm fan spin at low rpm (barely inaudible) moving air between the drives, over the surface of the drives and cooling the metal of the case as well.

[DC1MC]

And I've actually empirically proved that what actually dies it's the SSD controller

I am afraid that you did not prove anything. To prove that it is controller that died and only - you have to replace dead controller chip with working and run full diagnostics of SSD proving that flash chips are intact and controller is only part that failed. Did you do that? What if flash died and controller is fine, just unable to read it's firmware (from flash)?

One of the Sand Disk controllers crashed spewing magic smoke on few pins, the OCZ drive has Micron in TSOP flash that has a data-sheet and it can be accessed, sometimes when I'll feel like experimenting again I'll invest few euros in another bunch of cheap drives and gently solder away their flash.
The OCZ SandForce ST-1222TA3 PCB even had serial and a JTAG port, I could bet that interesting experiments cold be done there. Probably on SSDs now these ports are long gone, the SanDisks have branded BGA flash chips and totally no debug ports .
In any case, the little on-board power regulators are still working !!!, these are indeed resilient. I'll use the PCBs to train in removing the controller, maybe I can access the flash, it could be a business opportunity if the flash could be read after the disk has become invisible to the BIOS.
Also, even if locating the controller firmware in the data flash may sound like a good idea, I do think that it's put into some internal memory, those guys are paranoid with their IP, anyway until some more experiments will be done, this will remain just suppositions.

Right now I'm waiting my multi-port SATA cards and tomorrow I'll raid Saturn for some cheapos Crucial/WD to be used as boot disks and see if Ubuntu server knows about the multiport cards. Then I'll peel them off the plastic case and thermal-glue them on the metal cage.

Then the big question will show up:what should I replace the aging fans with, I want to keep them (not all 3, but at least one as emergency cooling) ?

 @mariush Many people on amazon's Crucial BX500 were mentioning that during normal usage it reaches internally around 75°C !!!, the 3-40 degrees are a (heat) pipe dream .
Before peeling off the boot disks to cool them properly, I'll try to see if I can read the internal temperature and do some bonnie++ tests with/without case and glued on metal cage.

 Cheers,
 DC1MC




 

[David Hess]

And I've actually empirically proved that what actually dies it's the SSD controller, it has nothing to do with the flash (didn't backed it directly at 95, I've done long other reliablility tests with lower temps, never to have a data integrity problem) , ...

Corruption of the Flash translation data structures is indistinguishable from controller failure which is why you can brick drives by corrupting the Flash translation data structures.

Writing data on the SSDs is what consumes power, and you'll only write for brief periods of time. Natural air cooling will take care of moving the generated heat.
When you're reading data from SSDs you're basically only dealing with the idle power consumed by the controller, which is what... 0.3w to 1w ... that's nothing.

I ran into this problem recently when configuring an SSD to use a SATA to USB bridge.  Power requirements can be higher than the 2.5 watt USB 2 specification or the 4.5 watt USB 3 specification even when only reading.

[Monkeh]

And I've actually empirically proved that what actually dies it's the SSD controller

You've proved what dies under extreme thermal stress is the hottest part on the board. This is not only not surprising, but not of any particular value.

[DC1MC]

...

Corruption of the Flash translation data structures is indistinguishable from controller failure which is why you can brick drives by corrupting the Flash translation data structures.
....

What is a "Flash translation data structure" and how it makes the controller not serponding to the basic SATA commands ?

[Monkeh]

...

Corruption of the Flash translation data structures is indistinguishable from controller failure which is why you can brick drives by corrupting the Flash translation data structures.
....

What is a "Flash translation data structure" and how it makes the controller not serponding to the basic SATA commands ?

It's the incredibly critical block of data which tells the controller exactly how and where data is stored. Without it, the controller cannot piece anything back together, fails to initialize, and does not talk to the host.

Remember, there's no direct relationship between an LBA as the host sees it and the content of the NAND. There's compression, encryption, wear levelling, striping, and presumably some parity.

[DC1MC]

And I've actually empirically proved that what actually dies it's the SSD controller

You've proved what dies under extreme thermal stress is the hottest part on the board. This is not only not surprising, but not of any particular value.

The goal was to see what actually dies and make the device totally invisible to the controller, causing instant 100% data loss, not gradual failure that may be recovered from with the help of RAID data redundancy.

So far the results are:
- The cheap disks have an exceedingly bad thermal management, besides the thick plastic cases that have partitioned inside in such a way that those compartments are not communicating in between them (see the attached picture of a SanDisk shell), the PCB and chips are not touching directly any of the case walls, so the head accumulates and transfer is as difficult as possible.
- The controllers are indeed the hottest part of the assembly and forcing them to go to 6Gb/s to have the marketing bonus points (useless for controllers without significant RAM cache) made them even more prone to failure.
- Still waiting to see what's actually happening when the flash have reached its limit of operations, will the disk go BOOOM and fully disappear form PC controller suddenly, will the SMART give early warnings and/or make device R/O as promised, will I have HDD like individual "bad sectors" errors or what ?
I plan to put a cheapie in the most comfortable operating conditions (cooled directly) and gently write the hell out of it, what it will actually happen in the end ?

 Cheers,
 DC1MC

[DC1MC]

...

Corruption of the Flash translation data structures is indistinguishable from controller failure which is why you can brick drives by corrupting the Flash translation data structures.
....

What is a "Flash translation data structure" and how it makes the controller not responding to the basic SATA commands ?

It's the incredibly critical block of data which tells the controller exactly how and where data is stored. Without it, the controller cannot piece anything back together, fails to initialize, and does not talk to the host.

Remember, there's no direct relationship between an LBA as the host sees it and the content of the NAND. There's compression, encryption, wear levelling, striping, and presumably some parity.

I've only experimented with the OCZ manufacturing tools, there is nice Fedora Linux image floating around  with the OCZ manufacturing tool (written in Python !!!) where you can play with everything manufacturing related, from serial numbers, firmware, capacity, secure erase and so on. The manufacturing pack available was for the SandForce controller, that all in all is a standard ARM MCU, not unlike an NXP iMX something. I've used it extensively and the firmware is (at least for that model ) saved in a separate serial EPROM, most likely a NOR device.
It also allows for selecting what NAND Flash type is connected to the controller, from a list of reasonable different types, and while you CAN confuse the controller by selecting wrong flash types or loading a bad firmware, it never goes in a state where it doesn't respond to basic SATA commands and can be pulled back from the grave. If it's too far gone, one needs to strap a jumper on the board to put it in "default firmware manufacturing mode".

In any case, to me it sounds smart and sensible to put the controller firmware and configuration in a separate flash from the data devices, either internal to the controller or a small SPI.

But I don't consider that will be under the dignity of some manufacturers to put it in the data flash to save some costs, but it seem unlikely. And remember we're not talking here about data errors, but about the device not being sensed by the controller at all.

  Anyway, while this is an interesting topic in itself (accessing the disk controllers via the debug ports and playing with the firmware) I'm just interested in building a somehow reliable NAS and I think I've got enough data to improve things a bit. Whis me luck with the multiport controllers if the seller will ever ship them.

 Cheers,
 DC1MC