AI centres

[Terry Bites]

I watched some very interesting YouTube videos on the consequences of the rapid increase in the number of AI centres.

Bot demand is skyrocketing. Generative AI centres will be interconnected and be able to aggregate their capabilities. That poses some new questions.

They differ significantly from server farms taking up vastly more energy. Like 100kW and higher per rack! A centre might have 100 racks, thats 10MW!

They need to be near big energy providers unlike the server farms which need to be close to communications hubs.
Infrastructure will struggle to keep up with demand if immediate action on power generation is not undertaken.

On the upside there’s never been a better time to buy stock in power distribution and cooling system companies.

[artag]

What will they do with them when the bubble's finished bursting ?

[golden_labels]

Let me start with a digression. Each time you see a phrase:

• In the form of “X uses Y of resource Z.”
• Y is an apparently a large number.
• The phrase frames it as a problem.

Let a warning light appear in your head. Regardless of whether there is a problem or not, the phrase itself is usually deceptive.⁽¹⁾ It’s widely used in politics and ideological narrative for a reason.

Returning to the main topic: it’s complicated™. Any oversimplified answer is likely wrong.

The most important thing to ask is: why is 10 MW used? It’s common to point fingers at an industry and say “they cause harm.” But it’s not like a rich, bored bastard sits on a pile of money and burns them for fun. Industry’s footprint is a function of consumers’ usage and they exist exclusively because there is a demand. So where this 300 GWh a workday goes to? What amount of energy would be needed for humans to do the same work?

10 MW is also very visible, because it’s concentrated. It’s easy to spot. There is a saying: “what you don't know can't hurt you.” Spread the same workload across a thousand smaller facilities and nobody cares, even if their footprint per unit produced is much larger. This doesn’t mean the energy usage in general is not a problem, but this must be taken into account.

Historically we have plenty of examples, where both hallucinated and real technological breakthroughs ramped up resources usage. Decades, centuries, millennia later we treat this as something normal. While this technology itself is not new, and qualitatively there is no great leap, the progress in electronics changed it quantitatively to the point it offers new features. Similar to other similar situations, I don’t think we should worry about the associated increase in energy demands. Not at this moment. A change is a short-term thing. What matters is how it plays out long-term. If it’s useless, if it doesn’t deliver, then indeed we should worry. But fortunately history shows in such cases useless technologies are starved of investment and die out. If it works, it will stay with us and it’s going to be just another normal technology and normal resource usage, as it always has been. Of course aiming for reducing the energy used will be important, but that’s going to be an issue after the situation stabilizes.

To wrap it up, what happens when the bubble bursts? Or “if,” depending on one’s convictions? How to squeeze most of it? For once, datacenters and large facilities of that kind are already built near power plants, which they supply with funds, or build their own energy sources. If they fail, we’re left with a lot of nice power infra built on their expense.

Discalimer: this post is about energy usage. Relying on conflict resources, social effects of smortnets, concentration of capital and other issues were not touched by Terry and I stay within the topic they set.

⁽¹⁾ It does not automatically imply the person using it is intentionally deceiving you. One may simply be repeating what they heard elsewhere. But it may also be a result of careless thinking or a complete lack of it.

[Andy Chee]

I have trouble with describing the current boom as a bubble that could potentially burst.

To use an example, let's look at the so-called dotcom bubble burst with 25 years of hindsight.  It should be self-evident, the so-called dotcom bust has not caused any diminished usage of the internet in today's economy.  As such, speculative investment in AI today, will not destroy AI in the future.

The attachment of resource usage to AI development can also be misleading.  Consider 100W of LED lighting compared to a 100W incandescent globe.  Both consume the same amount of electricity, yet the former produces more lumens per watt.  AI processors are also likely to make performance efficiency gains.

IMO, the biggest fear of AI is introducing human biases into the AI's training.

[Berni]

There will always be demand for datacenters and electricity.

The bubble bursts once the investors find out that all the hype that AI keeps promising is false and start panic selling their investments.

At that point AI will be everywhere and so the need for computation to run said AI will be very much in demand. But the demand for compute for training the AI will certainly drop a lot. But this compute runs on the same kind of hardware so it can be repurposed for future demand in inference. But safe to say that the demand for enterprise Nvidia GPUs will fall off a cliff and so will Nvidias stock price. The datacenters will scale back for sure, they might start temporarily turning off some GPU clusters while cloud GPU compute will get dirt cheap (since leasing a server with 0 net profit is still better than shutting it down and having it sit), maybe they might repurpose some for cloud gaming (Perhaps the next generation Xbox console might be just a thin client that runs games in the cloud).

The typical rule that programmers get more careless with resource usage as computers get more powerful also means that those cloud GPUs will get used up. They will be using AI to do computation that never needed AI in the first place, but someone way lazy and it was easier to implement with AI while using 1000x more compute in the process.

The datacenters themselves can also be repurposed. The servers in datacenters are constantly upgraded every so often (as they become obsolete), so they can always fill the racks with something else.

The drop in electricity demand is also something that could be used meaningfully for the push for electrical mobility like electric cars.

[golden_labels]

I have trouble with describing the current boom as a bubble that could potentially burst.

To use an example, let's look at the so-called dotcom bubble burst with 25 years of hindsight.  It should be self-evident, the so-called dotcom bust has not caused any diminished usage of the internet in today's economy.  As such, speculative investment in AI today, will not destroy AI in the future.

It all depends on what you mean by “AI.” In recent years this word has been abused by marketing departments, journalists, politicians, and commentators to such an extent that it became basically meaningless.

If you mean the branch of study, the associated algorithms, and in particular the advanced statistical methods that gained popularity in the 2000s: in this area there is indeed nothing to burst. It’s solid, stable stuff that has been steadily progressing for the past 20 years, anchored itself in many industries. As a part of modern mathematics and research methods it’s rather going to gain more land than lose it.

The extremely loud off-shoot, that grabbed population’s attention a few years ago, is a different story. Predicting bubbles is a task for astrology, in one or other of its forms, but it’s not that insane to recognize features something shares with known bubbles. This doesn’t mean it has to end the same way: after all there are many ways these patterns may dissapear. But the potential is there.

Bursting doesn’t mean that technology has to be abandoned or its products discarded. There are many brilliant engineers benefiting from the influx of funds, expanding knowledge, designing new solutions, and freely experimenting with this matter. Their work will not disappear.