idea: computing cluster on a circuit board

[clickcell]

Hi, new to the forum but a long time eevblog viewer.

I'm interested in electronics but essentially a complete novice. I wanted some thoughts on how feasible my idea is and how to turn it into a reality.

Basically I want a circuit board, no bigger than a PC motherboard, that would contain at least half a dozen SoCs and associated flash storage. Plus an ethernet switch that all of the SoCs would be connected to via the PCB traces. The board itself would have a single ethernet port and some kind of probably DC supply input. For booting I'm not sure, could a single piece of flash serve as a read only filesystem to all SoCs to boot from?

Idea being obviously to get as many compute nodes as possible as cheap as possible.

[Nerull]

Once you get this thing designed and built, I have a hard time seeing it being cheaper than something like this: http://coen.boisestate.edu/ece/raspberry-pi/

If you had the volume to bring the price down it would be interesting to see, though.

[c4757p]

I don't think it's practical at all, but I think it would be a fun design project. Quite complicated, though. What's your target for processing power?

[clickcell]

Once you get this thing designed and built, I have a hard time seeing it being cheaper than something like this: http://coen.boisestate.edu/ece/raspberry-pi/

If you had the volume to bring the price down it would be interesting to see, though.

I was hoping it would be cheaper due to ditching connectors, and packing nodes onto a PCB saving on ethernet cable and connectors needed to build a network. If you can't put 8 or 12 Raspberry Pi CPUs on a board and make it cheaper than 10 Raspberry Pis then everything I know about economies of scale must be wrong.

[clickcell]

of course you cant print twisted pair on a circuit board, but how much does that matter?

[ddavidebor]

The fact is there are more rasberry pi than your board

[8086]

Once you get this thing designed and built, I have a hard time seeing it being cheaper than something like this: http://coen.boisestate.edu/ece/raspberry-pi/

If you had the volume to bring the price down it would be interesting to see, though.

I was hoping it would be cheaper due to ditching connectors, and packing nodes onto a PCB saving on ethernet cable and connectors needed to build a network. If you can't put 8 or 12 Raspberry Pi CPUs on a board and make it cheaper than 10 Raspberry Pis then everything I know about economies of scale must be wrong.

Unless you can make the same quantity of these as they make of raspberry pis, so probably hundreds of thousands, economies of scale won't help you. Connectors cost pennies, PCB space (especially to the tolerances required) costs a lot more unless you have quantity on your side.

[c4757p]

of course you cant print twisted pair on a circuit board, but how much does that matter?

Parallel differential trace pairs are the PCB equivalent of a twisted pair. You run the traces close together and keep them both the same length (more professional PCB layout software can do this for you). You'd need to do that with any high speed digital signal, not just Ethernet.

[clickcell]

HDMI and USB connectors pfft I could save multiple pennies per computational node. In the future all super computers will be built my way anyway.

[Nerull]

Sure, a pure parts list will be cheaper, but you also have to figure in design time and software, board fab and population, etc. High frequency circuit design is no joke for a complete novice, so you're likely looking at contracting that out. The Pi can be as cheap as it is because they are made in high volume. The fab makes 30000 of em a month. There are over a million of em out there.

Look at the uCurrent. Very simple board - much simpler than a Pi, for sure. Costs over twice as much!

You're going to be looking at a lot of money to get one board, which will probably have something wrong with it. So start again at rev. B!

[homebrew]

I think it is the wrong approach altogether ...
What's the goal of the project? Or more exactly: What is the purpose of the cluster? There is no such thing defined as "THE SUPERCOMPUTER". What will your overall cluster architecture look like? Almost all clusters are built to purpose. For example, if communication-latency between the nodes is a problem, the higher end projects even use Myrinet or InifiBand instead of ethernet.

Regardless of what you do, you won't beat the server/cluster market in neither MIPS/Watt, MIPS/Volume nor MIPS/$ ... Not by reusing existing technology; the big manufacturers would have done it the same way already!

However there is some sort of web hosting project done with PI's:
Here these guys filled an entire Rack with PIs ...
http://www.raspberrypi.org/archives/3685

Pete

[sync]

What do you want to compute? Maybe Amdahl is killing you.

Regarding the file system. Use diskless setups for the computing nodes. A master server for providing the filesystems and controlling the nodes. Use two separated ethernet segments. One for the normal stuff and one dedicated for the computing intra node traffic. Btw: gigabit ethernet is slow. TCP/IP also.

This idea is not new. Search for "Blue Gene". But these are niche systems. Most problems doesn't run well on them.