solved: 8192-bit wide memory bus?

[incf]

I recently read the NVIDIA H100 GPU has a 8192-bit wide memory bus.

Has anything been published documenting how exactly they pulled off such a wide bus? (Special memory? A bunch of smaller memory modules in parallel with a complicated control scheme? etc.)

After a certain point I would imagine the "boring logistics" of having so many signals starts to become a challenge.

Googling and such leads me to believe that these GPU modules are so expensive and 'new' that teardowns and technical information are not as widely available as is for older architectures.

[edavid]

They use HBM stacks, which are 1024 bits wide:

https://en.wikipedia.org/wiki/High_Bandwidth_Memory

Also, it appears that H100 uses 6 stacks = 6144 bit bus.

[iMo]

So that’s why Micron moved out of the consumer business?

[incf]

I wonder if the DDR situation is mostly an issue of the bus type

ie. so maybe Intel and AMD motherboard chipsets just need to gain the ability to talk to memory that uses some version of this new HBC bus (I'm ignoring all of the logistical problems like HBC being for on-die communication and DDR for being for memory sticks)

[iMo]

..Micron said that the enterprise biz with those HBM is much more profitable, therefore they go out of the "consumer biz" (ie. Crucial finishes).

[MadTux]

Just incredible large micro-BGAs with shitloads of balls?
Certainly easier than doing 1kbyte/clock in serial mode.
Currently at 1Gbyte/s per pin, so that's just 8,192 Tbyte/s or 64Tbit/s with 8192-bit bus.
Kinda crazy, would be optical frequency, if just serial with no QAM or other magic

[Marco]

ie. so maybe Intel and AMD motherboard chipsets just need to gain the ability to talk to memory that uses some version of this new HBC bus (I'm ignoring all of the logistical problems like HBC being for on-die communication and DDR for being for memory sticks)

Interposers are expensive, even organic ones.

NVIDIA is moving to LPDDR5x for their CPU memory, almost as efficient in joule/bit, but can be done on standard PCBs and is fast enough. Save the HBM for the GPUs for now.

Intel&AMD have more inertia and are more bound to classical motherboards with memory slots, but it's starting to get very costly power wise. Unless there comes some revolutionary chipscale compressed attachment standard which can directly compete with soldering, all memory will be soldered soon.

[ejeffrey]

https://en.wikipedia.org/wiki/CAMM_(memory_module) is an attempted standard for lpddr modules. 

[Marco]

https://en.wikipedia.org/wiki/CAMM_(memory_module) is an attempted standard for lpddr modules.

Framework/AMD couldn't make LPCAMM2 work for Strix Halo. SOCAMM2 seems a lot less space wasting, maybe it's close enough to soldered, dunno.