list of good and bad miniPCIe and PCIe sATA and SAS controller for Linux

[DiTBho]

as kind of "hardware aggregator" topic, let's list good and bad HBA 

[DiTBho]

Marvell 88SE9215

This chip is a four-port 3/6 Gbps SATA Host Bus Adapter that provides a one-lane PCIe 2.0 interface and sATA,
          PCIe 2.0 x1 4-port sATA 6 Gb/s
and it's available in both miniPCIe and PCIe solutions.

On Linux, it only requires
          CONFIG_ATA
          CONFIG_SATA_AHCI

so it's served by drivers/ata/ahci.c as class 010601, means it's a common sATA-AHCI HBA

Does anyone have any positive/negative experience with this?

[tunk]

Support for some cards have been removed in RHEL 8 (and I guess 9):
https://access.redhat.com/discussions/3722151

[DiTBho]

RAID vs HBA SAS controllers: what's the difference? Which is better?

Next question: which HBA is non-x86(1) Linux-friendly?
(not easy to answer)

(1) { arm, arm64, mips, mips64, hppa, ppc, ppc64 }

[BradC]

as kind of "hardware aggregator" topic, let's list good and bad HBA 

Waste more resources because you're too lazy to use a search engine : https://forums.servethehome.com/index.php?threads/lsi-raid-controller-and-hba-complete-listing-plus-oem-models.599/

[Ed.Kloonk]

Just promise me that you'll confirm that the SATA cable is up to scratch before you deem a controller or driver as dodgy. Otherwise, I don't have a dog in the hunt. 

[DiTBho]

Just promise me that you'll confirm that the SATA cable is up to scratch before you deem a controller or driver as dodgy. Otherwise, I don't have a dog in the hunt. 

I am testing several { PCI, PCie, miniPCIe } HBAs on weird GNU/Linux platforms.
Spent 15UKP for SilverStone SST-CP11-300 premium Low Profile Ultra Thin 90 Degree 6Gb/s SATA bables 
(yup, 15UKP each cable)

[Ed.Kloonk]

Hey, big spender.

 

[DiTBho]

The list of LSI HBA is a nice thing to have, but megaRAID (as well as others, including Marwell') has some quirks and issue with
- 64bit kernel PCIe-64bit transactions where not correctly supported (e.g. RPI, ...)
- PCI-BAR wants laaaaaaaaaaaaaaaarger size

need to buy more HBAs for testing/mastering all of these technologies   

crazy, one of my testing machine is this

Code: [Select]

____________________________
| can do hw-RAID            |
|                           |
|     HBA     PPC440       SAS --------- 4x sATA DISK
|             512MB ram     |
|             8M flash rtOS |
|___       _________________|
    |      |
    | PCIe |
    |      |
   |PCI-to-PCIe   |
___|______________|_________
|                           |
|   GNU/Linux               |
| software kernel raid      |
|                         uart
|     SBC     PPC40x        |
|             128MB ram   eth0 ------ sshfs
|             ramrootfs     |
|___________________________|

actually the HBA has more clock, ram, and horse power than the SBC 

Accepting 20MB/sec, it is enough to run kernel software RAID-{ 0, 1, 5, 6 } without any advanced file-system { ZFS, BTRFS} on it.

it does not make any sense, done *** because science ***

[nomead]

miniPCIe probably limits the choices to ASMedia and Marvell SATA offerings. As a "dumb" controller ASM1061 seems to work decently on x86.

Your setup looks like quite Commodore 64'ish. Mass storage being even smarter than the host. 
Is that PCI-to-PCIe conversion done with chip from ITE or Pericom?

[DiTBho]

miniPCIe probably limits the choices to ASMedia and Marvell SATA offerings. As a "dumb" controller ASM1061 seems to work decently on x86.

yeah, DeLock has some interesting miniPCIe products

DeLock 95233 -> Asmedia ASM1061 dumb controller
DeLock 95264 -> Asmedia ASM1061R (needs to be re-programmed), offers { SPAN, hw-raid0, hw-raid1 }
DeLock 95000 -> Marvell 88SE9215 (needs to be re-programmed) and it has SFF-8087 female connector

cost { 25, 50, 90 } euro: I'm going to buy some samples 

Your setup looks like quite Commodore 64'ish. Mass storage being even smarter than the host. 
Is that PCI-to-PCIe conversion done with chip from ITE or Pericom?

PCI-to-PCIe conversion done with chip from ITE ('cause was the first thing found around on eBay) 

[BravoV]

Ok, I'm cheap  , but I love server's grade stuffs, and luckily I was able to score these babies cheaply, and I got two identical, one as backup for my storage stuffs.

Bought used ones, as just can not afford the new one, as each cost a kidney. 

Its Dell PERC H730 1GB cache with on board battery backup RAID controller,  8 ports 12Gbps PCI Express (PCIe) supporting 3Gbps, 6Gbps and 12Gbps SAS or SATA hard-disk or solid-state drives.

[DiTBho]

Its Dell PERC H730 1GB cache with on board battery backup RAID controller,  8 ports 12Gbps PCI Express (PCIe) supporting 3Gbps, 6Gbps and 12Gbps SAS or SATA hard-disk or solid-state drives.

I think something similar, brand new, costs in the range 1200-1500 euro 

there are some suspiciously too-cheap controllers on Aliexpress and eBay/China for 50-100 euro (WHAT?!?)

but, if no matter how you get your RAID controller and how much you pay for it: will the Linux kernel v6 able to manage the "hardware RAID" engine on non-x86 machines?

This is the main question ... and usually the answer is ... no

On { HPPA, MIPS, PPC, SH } you can't use the x86-BIOS-extension mechanism, therefore the MEGA-RAID cannot use re-configuring functions, written in x86 assembly.

When you are lucky, you can temporarily plug the card into a x86 Windows/Linux SBC to make good use of the BIOS extension mechanism and re-configure the controller. Usually a text interface appears during booting, and you can scroll and press the enter-key or space-bar to confirm settings.

This, if and only if, there is a NVRAM able to keep the configuration, the battery is OK and the controller doesn't need to automatically reset the NVRAM if it doesn't find an x86-BIOS-extension environment.

plus, sometimes you also get GNU/Linux x86{32, 64}bit binaries as RAID utilities to run-time interact with the controller.

Two possible solutions

• add an x86 emulator in userspace with a door opened in kernel space to execute such a x86 code
• reverse engineer the MEGA-RAID BIOS-extension and x86-utilities to understand which/how registers you have to configure

why is life so complex  ?

[alm]

You should be able to find hardware RAID controllers for cheap. Server CPUs have gotten so powerful that there's little performance benefit, and storage technology has moved on from the old RAID levels. For large scale storage people use ZFS, and for performance they use PCIe SSDs. That doesn't leave much room for hardware RAID.

How are you going to boot from these controllers if you can't run their BIOS?

[DiTBho]

How are you going to boot from these controllers if you can't run their BIOS?

Code: [Select]

.              |--------|
               |        |
               | config |
               |        |
               |--------|
               |        |
swhw_reset --->|        |---+
               |routines|   |
exceptions --->|        |   |
               |        |   |
               |--------|   |
               |        |<--+
               | u-Boot |
               |        |---+
               |--------|   |
               |        |   |
               |        |<--+
               | kernel |
               |        |-------> load(ramroots,CF0)-----> /sbin/init -> { boot-scripts { kernel drivers }, { Apps } }
               |--------|
                ROM{0,3}

my hw doesn't to boot from the HBA. The kernel is loaded from the ROM (flash, 16MByte), while the rootfs is loaded from a CompactFlash (32Mbyte, MUSL-stripped), and, once in running in ram, the kernel can load the HBA kernel module, etc ...

Currently the best result has been obtained with a four channel SIL PCI-HBA, without any BIOS extension, so it is pure kernel software RAID.

PCIe-HBAs add more complexity for me, and my MEGA raid controller has been lobotomized and re-programmed - getting rid of 90% of its firmware - to act as a dumb HBA.

Which is ... pure shame, but at least it works ... just, I still cannot make good use of its large 512 Mbyte ram, which is mostly (only 8 of 512 Mbyte used) unused with the controller in "dumb" mode, but I am trying to re-map it as system ram so I could "expand" the limited physical ram I have on the PPC40x SBC.

I feel it as frustrating waste of precious resources, which, as much as I try to come up with tricks ... I haven't succeeded yet to make good use.

---

The other test machines are native PCIe with 8 Gbytes of RAM, 1.1Ghz CPU, and boot from a SCSI disk. Again the sATA HBA kernel driver is loaded by a userspace script once the kernel is already booted and running.

[BravoV]

I think something similar, brand new, costs in the range 1200-1500 euro 

there are some suspiciously too-cheap controllers on Aliexpress and eBay/China for 50-100 euro (WHAT?!?)

Yep, its expensive when new.

I got mine both for about < $250, and it was quite an effort and needed some detective works involved, like querying/phone calling at local companies that do computer's scrap, as these kind of server stuffs most were write off from big companies, like those banks, and this kind of controller usually used in small or mid sized server, used in branches, and once the company did a periodic so called technology refresh, these stuff are usually thrown away as scraps.

As I said, to procure this kind of stuffs, needs some effort, at least for me here locally.

but, if no matter how you get your RAID controller and how much you pay for it: will the Linux kernel v6 able to manage the "hardware RAID" engine on non-x86 machines?

Sorry, can't offer much help here on non x-86 machines, as I use it in x86 platform.