AI Is Stress-Testing Its Own Physical Infrastructure
Editor's note: AI's momentum isn't slowing down anytime soon...
That said, it's facing another bottleneck: Due to the sudden spikes in power consumption, the physical components of data centers are starting to degrade faster than anticipated. And with AI demand still in full swing, companies will do whatever they can to keep their models running.
Today's Masters Series was originally published in the August 26 issue of the free Altimetry Daily Authority e-letter. In it, Joel Litman – chief investment officer at our corporate affiliate Altimetry – analyzes the newly forming cracks in the AI sector and highlights the kind of businesses that will be crucial for the data-center build-out...
AI Is Stress-Testing Its Own Physical Infrastructure
The amount of power needed by a data center can change greatly – and quickly...
When a large AI model is being trained, hundreds of thousands of graphics processing units ("GPUs") can dramatically change their power consumption – all in about a millisecond...
The result is a violent surge in electricity demand.
Bloomberg reports that AI facilities can briefly draw as much as 50% more power than their designed capacity in a split second. For instance, a 1-gigawatt facility could suddenly demand roughly 1.5 gigawatts.
And some planned U.S. AI campuses are more than five times that size.
Those power swings are already taking a toll on data centers... Gas turbines are developing cracks. Components inside smaller generators are breaking. And batteries installed to smooth power flows are wearing out within weeks.
AI's infrastructure requirements are forcing data-center operators to rethink everything. In turn, this is creating a major investment cycle in the equipment needed to keep AI computing power running reliably.
The chip boom is becoming a power boom...
BloombergNEF estimates the power demand associated with AI chips will climb to more than 300 gigawatts by 2033. That estimate includes the computing hardware itself along with the networking and cooling infrastructure needed to support it.
The U.S. is expected to account for 64% of new AI electricity usage between 2022 and 2033.
That means the industry needs to build enormous amounts of new power infrastructure.
And making sure that equipment doesn't break down adds another layer of spending.
Traditional data centers have generally consumed electricity in a much steadier pattern. Meanwhile, AI training can send huge blocks of computing capacity on line and off line almost instantaneously.
The equipment connected to these facilities must be able to sustain those surges without losing power quality or damaging the hardware.
That's already proving difficult...
At xAI's Colossus facility in Memphis, Tennessee, gas-fired turbines reportedly developed cracks due to these power fluctuations. Batteries were added to help absorb the fluctuations and protect the spinning equipment.
Similar problems have appeared elsewhere. Some batteries used to stabilize data-center power have needed replacement far earlier than expected because of the constant strain.
It's clear that reliability is becoming a significant part of the AI build-out...
That puts a premium on equipment designed to control electricity rather than to simply produce more of it.
Data centers increasingly need hardware that can smooth rapid power movements before they can damage sensitive equipment.
The need for consistent cooling is equally important.
As AI racks become more power-dense, more electricity is concentrated inside a smaller physical area. Keeping those systems within safe operating temperatures becomes increasingly vital as operators pack more computing capacity into every facility.
AI data centers are designed around continuous operation. If this computing infrastructure goes off line any time there's a sudden fluctuation in power demand, the financial damage extends far beyond replacing a failed component.
Some facilities are reportedly achieving uptime closer to 80% of the time – even though they were built to run continuously.
Nvidia (NVDA) has already responded to these issues... The chipmaking titan has been focusing more heavily on increasing the efficiency of power delivery throughout data centers.
The infrastructure around AI has become more critical than ever...
Chips remain the heart of the AI build-out. Yet every dollar poured into strengthening computing capacity raises the value of keeping that capacity on line.
A multibillion-dollar AI facility can't produce revenue when its power systems fail. As data centers grow larger and workloads become more demanding, operators will have stronger incentives to spend heavily on the equipment that protects those investments.
That expands the AI capital-spending cycle far beyond semiconductors.
Power-management systems, electrical equipment, and backup generation are moving closer to the center of the AI investment story.
The next phase of the AI capital-expenditures cycle belongs to the infrastructure that keeps the data centers running around the clock.
Regards,
Joel Litman
Editor's note: Elon Musk has had ventures in electric vehicles and satellite communications. Now he's setting his sights on AI. With bottlenecks forming on multiple fronts, he's seeking to fill the voids and become the "Kingmaker" of the sector.
This is building an incredible opportunity for investors. Musk will need to rely on select industries to make his dreams a reality. And Joel says that one signal indicator can reveal which ones will be the big winners. But time is running out.
To hear Joel's full warning and see where he says you should put your money, click here.