You can buy small NAS PC cases that have 4 hotswap bays on the front and are about the size of a larger shoe box. Due to these small size only fit the mini ATX boards (the smallest common ATX form factor) but you do still get 1 PCIe slot on it. You typically get a tiny ATX spec PSU included with the case (Just make sure it is a known brand).
I can go find some photos of mine later. I bought such a case from some random EU web shop, put in in a 7th Gen i5 CPU and filled it up with 4 of the 4TB WD Red drives along with a 250GB NVME SSD as cache. The thing runs Unraid as the OS.
Where things got tricky is when i tried to make it as quiet as possible too. So it involved modifying a giant CPU heatsink to fit inside it, 3D printed ducting to make the CPU fan do the job of the case and PSU fan. Working in these small form factor cases is also annoying because things are so tight and you often need to take things apart completely to get to some parts. Later on i also added in a SAS HBA card to expand the storage capability with external drives.
Hotswap bays are not strictly required but are nice to have since in the case of a dead drive it allows easy swapping of a drive to start rebuilding the RAID as soon as possible without accidentally knocking out more drives or shutting down the machine by messing with cables inside. The OS is a matter of taste. You got TrueNas, FreeNas, Unraid..etc
If you need high performance the ZFS RAID is the way to go. If you want more resilience and easy expandability Unraid is the way to go. You can't easily resize ZFS and a corrupted ZFS array is difficult to recover. In Unraid you can just add any size drive to the array and simply have to recalculate parity data, if the Unraid array corrupts to the point of being unrecoverable you can still just take out drives one by one and read a normal linux partition on them, so you only loose data on drives that died, not the whole array. However due to this you don't get any RAID performance boost. Reading the array is only as fast as the read speed of a single drive while writing is only half the drive speed (but this can be boosted by the SSD cache). For most home use Unraid is typically still plenty fast enugh, but i would avoid it for a corporate environment with lots of concurrent users.