Wafer scale chip

[Mr. Scram]

Apparently a company has developed a chip that encompasses almost an entire 16nm wafer. The 46,225 mm² chip supposedly has 1.2 trillion transistors and 18 GB on-chip SRAM and comes with 400 cores. Even if we ignore the buzzword rich announcement opting for a wafer sized chip is definitely an interesting choice. I can't help but wonder how you'd power and cool such a massive chip effectively.

https://www.cerebras.net/

[edy]

What is the yield of such size chips? I'm not sure about the latest stats but I was under the impression that there are defects that occur and some chips don't pass tests. If you now "put all your eggs in one basket" by integrating such a large piece of silicon, does the entire thing have to be perfect? Or do they have redundancy built in so they segregate parts and inactivate or mark bad areas not to be used?

[Mr. Scram]

What is the yield of such size chips? I'm not sure about the latest stats but I was under the impression that there are defects that occur and some chips don't pass tests. If you now "put all your eggs in one basket" by integrating such a large piece of silicon, does the entire thing have to be perfect? Or do they have redundancy built in so they segregate parts and inactivate or mark bad areas not to be used?

That's exactly what I was asking myself. The center of a wafer tends to be better developed than the outer edges too so I can only assume some form of compensation or redundancy has to be built into the design.

[SiliconWizard]

Well, they are still not using the whole wafer area (wafers are usually circular), so they are excluding the outter edges anyway.

I have no clue about the yield though. The cost of just one of those chips must be pretty gigantic, and yes cooling it down would require something pretty massive. I don't really know about the practicality of it all beyong the wow effect.

Yes they probably have embedded means of testing each core and each RAM block individually and disable the defective ones instead of having to trash the whole wafer if just one block is defective.

Still. A single of those chips must cost like hundreds of thousands of dollars, especially at the quantities that this would be produced.

Given the yield considerations and the unique challenges (for cooling, for packaging...) that such a design would pose, I don't even know how cost-effective or how reliable a system designed around that would be.

[filssavi]

As far as I have understood it is not a single massive chip but rather few dozens/hundreds chips on the same substrate, that have not been diced, with wire bonding in between (i immagine), this not only confines potentially destructive defects to a single ic and not the whole wafer, moreover I suspect that using standard lithography machines there are single die sizes limit, so even if they wanted to the could not do a single wafer device*

also from the anandtech livestream there are some pictures of slides were they talk about pervasive on chip networks, thus they have even finer granularity to disable broken cores


*wafer level devices are routinely used in high power switching elements (think a 6 kV 2kA Thyristor), where a 100 mm wafer is used for a single switching element, the silicon purity required in that case is extremely high, even higher than for cutting edge processes, as a single defect will kill an entire wafer (and with such high blocking voltages the smallest contamination is enough to render the whole device a semiconducting firework waiting to go off)

[Mr. Scram]

Well, they are still not using the whole wafer area (wafers are usually circular), so they are excluding the outter edges anyway.

I have no clue about the yield though. The cost of just one of those chips must be pretty gigantic, and yes cooling it down would require something pretty massive. I don't really know about the practicality of it all beyong the wow effect.

Yes they probably have embedded means of testing each core and each RAM block individually and disable the defective ones instead of having to trash the whole wafer if just one block is defective.

Still. A single of those chips must cost like hundreds of thousands of dollars, especially at the quantities that this would be produced.

Given the yield considerations and the unique challenges (for cooling, for packaging...) that such a design would pose, I don't even know how cost-effective or how reliable a system designed around that would be.

They built in various forms of redundancy and the ability to restructure the grid.



Apparently they cool the chip with "perpendicular" water cooling.


Finally the ridiculous size of this thing.



https://www.anandtech.com/show/14758/hot-chips-31-live-blogs-cerebras-wafer-scale-deep-learning

[nimish]

This thing rules. I'm glad someone just decided to throw enough $ at silicon fabricators to make this real even if it's likely going to fail in the market.

They plan on power it via perpendicular copper wires (since grounds plans don't offer enough power handling!) and liquid cool in a similar manner

15kW power draw, and god knows what kind of custom server chassis to support it.

If you can keep it occupied it's likely far more energy efficient than networking 100x GPUs or TPUs or whatever. pJ/bit cost increases immensely when moving off chip.

[Mr. Scram]

This thing rules. I'm glad someone just decided to throw enough $ at silicon fabricators to make this real even if it's likely going to fail in the market.

They plan on power it via perpendicular copper wires (since grounds plans don't offer enough power handling!) and liquid cool in a similar manner

15kW power draw, and god knows what kind of custom server chassis to support it.

If you can keep it occupied it's likely far more energy efficient than networking 100x GPUs or TPUs or whatever. pJ/bit cost increases immensely when moving off chip.

Regardless of whether it'a practical or a success in the market I'm glad someone implemented what's on the surface a rather silly idea. The 15 kW number is quite remarkable too and will require some planning to pull off.

[Circlotron]

Windows Eleven will still probably take 5 minutes to boot on it. 

[Mr. Scram]

Windows Eleven will still probably take 5 minutes to boot on it. 

If the regular 10-15 seconds isn't quick enough you can reduce it to under 5 seconds if you must. I doubt you'll get it to boot on this monster at all though.

https://www.tomshardware.com/reviews/fastest-windows-10-boot-time,5810.html

[nctnico]

What is the yield of such size chips? I'm not sure about the latest stats but I was under the impression that there are defects that occur and some chips don't pass tests. If you now "put all your eggs in one basket" by integrating such a large piece of silicon, does the entire thing have to be perfect? Or do they have redundancy built in so they segregate parts and inactivate or mark bad areas not to be used?

That's exactly what I was asking myself. The center of a wafer tends to be better developed than the outer edges too so I can only assume some form of compensation or redundancy has to be built into the design.

AFAIK they already do this on DRAM memory chips already. There is a mechanism to map bad blocks away and replace them with spare blocks.

Windows Eleven will still probably take 5 minutes to boot on it. 

If the regular 10-15 seconds isn't quick enough you can reduce it to under 5 seconds if you must. I doubt you'll get it to boot on this monster at all though.

That is just a boatload of crap. When Windows shows the desktop it isn't half way done loading everything. Microsoft just made that possible as eye candy and to boast another meaningless number but meanwhile the OS isn't ready for use at all when it shows the desktop.

[Mr. Scram]

AFAIK they already do this on DRAM memory chips already. There is a mechanism to map bad blocks away and replace them with spare blocks.


That is just a boatload of crap. When Windows shows the desktop it isn't half way done loading everything. Microsoft just made that possible as eye candy and to boast another meaningless number but meanwhile the OS isn't ready for use at all when it shows the desktop.

Building in redundancy is done a lot. I know what was the ATI did this in certain video chip families when they moved to a new node. That way they ensured they could get functional chips out of an imperfect process and managed to inch ahead of Nvidia which was struggling with the same node at that point.

Let's drop the Windows discussion. We've already got too many other threads for that and this will not be one of them.

[Circlotron]

^^ Yes. The fact that I said Windows Eleven, which doesn’t actually exist, should have made it apparent that it was just a passing joke, not a serious contribution to the discussion. Sorry, gents.

[helius]

Wafer-scale integration has been a dream since the 1970s (there is even a chip maker that named itself "Waferscale" but makes little eeproms and things). The yield problem has always been the big killer with these schemes. Yield is a function of the reciprocal square of die area (because the probability of each die having a defect on it is proportional to die area, and the number of dice you have on a wafer is proportional to the reciprocal of the die area), so chips stop being practically manufacturable at ~600 sqmm, never mind the ridiculous 46,000 sqmm in the OP.

The second major problem is that the scribe lanes between the dice do not join up exactly, because the carriage on the stepper cannot be precise to a nanometer as it is a mechanical system. That's a hard no, it simply can't be done. What you end up needing to do is to have a communication method that does not rely on joining silicon wires; you could try to build half a capacitor on each side of the scribe lane and use capacitive signalling, or expand the mask for the 3rd metal layer and hope the metallization lines up over the whole wafer, or do some kind of laser wire bonding. What is common to these approaches is that they do not have a real good likelihood of carrying sufficient power across the whole structure. That means that you need separate power and ground connections to each die—at that point, it's questionable if it is really wafer-scale integration or simply separate dice that are packaged without singulation. And indeed, the slide above shows only separate I/Os onto a circuit board.

This is nothingburger...

[Kleinstein]

They somehow missed an important point for the chips: they need pins for power to come in.  Cooling may also get difficult at that size - at least for high power density use, like a fast processor.

The interconnections between the units are a little difficult, but not a real no go. The structures just have to get larger so that they suit the stepper and alignment accuracy. So maybe 1-10 µm structures for the long range interconnecting metal layer(s).

[m98]

Somehow, I have the feeling that the engineers who where capable enough to design such a chip already thought of the interconnection, power distribution and cooling issues...

[NiHaoMike]

It looks like the holes could be to fit tubes for injecting liquid refrigerant into the cooling block and the vapor is carried off using a large pipe on the reverse side. Make the cooling block out of copper (probably the most sensible material to use, although silver would offer a bit of an edge where budget is plenty) and it will double as one of the power connections.

[SiliconWizard]

Maybe.

I'll be interested in seeing an actual and working system built around this though, and not just shiny pictures of a big "die".
 

[coppice]

Wafer-scale integration has been a dream since the 1970s (there is even a chip maker that named itself "Waferscale" but makes little eeproms and things). The yield problem has always been the big killer with these schemes.

Nope. The wafer scale schemes that have been tried have always been ways to map the good bits of the silicon, and join them all up to make the maximum amount of usable stuff. Some have even done this dynamically, so that after a failure you could switch off and on again, recharacterise the working bits, and have a working but slightly lower performance device. The problem always seems to be selling devices that you can't characterise as having a specific amount of storage or processing power or whatever the wafer is built to offer. It will always vary considerably from wafer to wafer. The customers mistrust such devices, and the makers hate how much it reveals about the strengths and weaknesses of their fab operations.

[SeanB]

Even Sinclair gave up on this approach, and he was really all for exotic cutting edge stuff that sort of worked, at a low cost. You might be better off just having individual separate large dice, and make a massive multilayer ceramic substrate to carry them all with BGA connections, and then put a back plate on, probably using IBM's copper plate technology to get a good thermal bond to silicon with a thick plating, and then a lowish melting point silver solder to bond a heat spreader to each chip instead. Otherwise, with this approach probably half the area of your wafer will be occupied with routing connections, which was the major killer for the original wafer scale integrated units, and as well a good part of the INMOS Transputer limitations. Power would have to be done with a lot of stitched bond wires or a separate set of precision made kovar power bus units that are either wire bonded down to each section or which are soldered to large power blocks that have hundreds of microballs on them for soldering.

[duak]

In the late 80's I used some Innova 128K * 8 SRAMs for an embedded 80386.  If memory serves, there were four non-singulated dice in the standard JEDEC package with redundancy to cover hard failures.  Innova apparently found some design fault and recalled and replaced the few hundred pieces we had bought.  My memory design had parity checking and I never had any trouble with them.  I think I might have a few in my debugging spares.

Other semi companies introduced simiIar parts with a single die within a couple of years so with only four dice Innova would have had a market advantage for two or three years.  It's like that when surfing the wave of Moore's law.  I  think the company folded after a few years and I can't seem to find any net presence.  I wonder if anyone else did anything like this.

I suppose if Moore's law doubling time increases then multiple attached dice and waferscale will become more important as a way to get maximum density.

[ddavidebor]

This thing has an absurd bandwidth, 100PB/sec !
Considering AI is all state and bandwidth, this thing will have crazy performance. Might even kick google tensor IC's ass...
Question is, are they going to sell enough of them? because they need a large cloud provider as a customer to make this profitable, and google is never going to buy them. Amazon, Alibaba and Microsoft are probably the only big enough remaining,

[Red Squirrel]

That's pretty interesting. Always been curious if a full wafer could be used as a single chip.  I could picture these being packaged either as a ridiculously huge BGA or just a COB on a card that can be inserted into a backplane and you have many of them. It would probably require a huge heat sink that spans the whole card.  Could be useful for insanely high density computing for a dedicated task.   I can hear the high pitched noise of the fans just thinking about it. 

[coppercone2]

doesn't this thing have horrible vibration resistance ?

how do you do proper low impedance planes?

people already hate ceramic PCB

[mac.6]

golden wafer are a thing, ie wafer where 100% of parts are functional. But they are excessively rare to say the least.
I know that one was produced in Intel Israel factory in the early 2k, never heard of another one since...