Liquid cooling is a pain for maintenance
Why do you say this?
I recently worked on the Leonardo supercomputer, small consultancy on a software face, not even important, anyway I had the opportunity to see how a supercomputer is made up of several nodes installed in different racks, and how each note can talks to every other node!
node[]={ A, B, C, D, E }
link[]=
{
A-B, A-C, A-D, A-E, // fiber-switch#1, the connection is high0speed, low-latency, unidirectional
B-A, B-C, B-D, B-E, // fiber-switch#2, "
C-A, C-B, C-D, C-E, // fiber-switch#3, "
D-A, D-B, D-C, D-E, // fiber-switch#4, "
E-A, E-B, E-C, E-D, // fiber-switch#5, ""
}
This thing scares me because I literally go into crisis when I have more than 5 nodes to connect, in the sense that, with hobby technologies .... you only have ethernet switches, and connecting each node with all the other nodes, means creating a bundle of fiber cables on fiber switches that never have enough ports, and in any case you end up with towers of fiber switches that you don't even know where to put ...
That's why we we do not create node-to-node connections of that type, but simply networks and subnets that do their job, cost less, take up less space, cost less, at the compromise of adding a lot of lag due to packet routing
Poor-man ethernet 6 ports switch: 5 for nodes + 1 extra port to allows you to route network packets to another group of nodes connected to another switch
switch ---------------
| | | | |
| | | | |___A
| | | |_____B
| | |_______C
| |_________D
|___________E
(dirty cheap, dirty simple)
If you try to realize with hobby/consumer technology the solution used in Leonardo, everything quickly gets out of control ... lucky them that they have dedicated things ... even liquid cooling systems with dedicated rooms, and, inside each node, copper heat exchangers, one for each processor, be it CPU, NPU, GPU, or whatever (from IBM zServer tecnology).
That's amazing! Each heat exchanger has copper pipes, one to let cold water in, one to let hot water out, and all the pipes are collected in special concentrators under the floor, which go directly to the floor below, where there are giant pumps (1.8m tall, taller than me!!!) that bring hot water to huge radiators outside the building.
This stuff works without any problems, and even Google/Switzerland, which I visited, has systems like this.
But, should it be a problem to make something like this small for a simple two node SBCs?
And if yes, what would be the "pain" for a maintenance "poor" man?