Artificial intelligence is forcing data centers to level up. As demand for AI applications continues to rise, operators are squeezing more computing power than ever into racks and racks of servers, which is driving up energy consumption and generating more heat.
Aside from raising sustainability concerns, these dense, next-generation AI data centers are putting pressure on every piece of hardware inside them to deliver better computing efficiency and performance to combat energy’s rising price tag.
“Data centers are no longer just about storing data,” Jae Ro, head of marketing at SIGNAL+POWER, a manufacturer of power cords and electrical plug adapters for business customers — including data centers — told The Forecast.
“They are about providing the massive computing resources that AI models demand. These facilities are evolving to handle the increased computational load while maintaining high levels of performance.”
And yet, these powerful AI data centers are not islands operating on their own. Along with smaller edge data centers and far-edge compute devices, they’re part of a broader ecosystem in which data is processed closer to its original source to create not only increased efficiency, but also increased capability.
Power-Conscious, Memory-Centric Computing
Because data centers and the servers that constitute them have traditionally been processor-centric, it’s not surprising that AI has spawned more powerful CPUs and GPUs that can ingest data more quickly. In the latest AI data centers, however, memory — not processing speed — is the key to eliminating performance bottlenecks.
Consider recent research by IDTechEx, which found increased demand for memory solutions like high-bandwidth memory (HBM), DDR DRAM and NAND, among others, as more data centers hit a “memory wall” that limits their performance.
The memory bandwidth that’s available at data centers will drive decisions about how much they invest in computing infrastructure to drive more efficient operations, said Praveen Vaidyanathan, vice president of DRAM process integration at Micron Technology.
“We are moving from a compute-centric infrastructure to a memory-centric infrastructure,” Vaidyanathan said in an interview.
But performance is not the only consideration. For every new iteration of DRAM, reducing power consumption also remains a goal, as denser memories and powerful processors consume more energy and generate more heat, which because of cooling requirements can have both environmental and economic consequences for data centers.
Micron’s sixth-generation 1-gamma DDR5 memory is one example of how memory makers are seeking performance gains while also keeping a lid on power. The 16-gigabyte DDR5 memory offers speeds that are up to 15% faster than its predecessor while also reducing power consumption by 20%.
Data Center Cooling is a Hot Topic
Energy efficiency has always been a challenge for data centers. It has become even more important, however, in the age of AI.
“As AI and machine learning models become more complex, energy consumption per task can be up to 10 times higher,” Ro said. “Data centers must adopt cutting-edge solutions like liquid cooling and two-phase immersion cooling.”
There’s no shortage of companies tackling the cooling problem. Some solutions focus on cooling down the system directly while others work indirectly by removing heat. In all cases, processors are a major target because they’re a primary heat source: Every voltage or current conversion leads to lost efficiency, which manifests as heat. As a result, power management is intertwined with cooling, which itself requires energy.
Until a few years ago, air cooling was the norm, according to Paul Konsbruck, CEO of LuxConnect, a sustainable data center operator in Luxembourg. Recently, however, Konsbruck has observed a shift toward water cooling.
“That technology has made huge steps forward,” Konsbruck told The Forecast.
Data center operators and chip manufacturers alike are exploring every possible option to mitigate heat generation in operations, which in turn reduces the energy burden for cooling.
“Innovative new chip designs can [remedy] this, as can liquid cooling, which also introduces changes to data center designs that were unheard of just 10 years ago,” Michael Skurla, co-founder of IoT software company Radix IoT, told The Forecast.
Moving data to where it will be processed can also reduce pressure on data centers, according to Ro.
“Edge computing plays a key part in the future of data centers by processing data closer to where it’s created, which helps reduce delays and speeds up decision making,” he said. “This is especially important for AI applications like self-driving cars or smart city systems that need quick responses. When combined with 5G networks, data centers can operate even faster, making everything run more smoothly and efficiently.”
Decentralization Deepens the Edge
Instead of the biggest data center operators, some companies are pushing their AI workloads to smaller regional data centers that are closer to the edge, according to Edward Berman, CEO of Atlantic Vision, an independent supplier of data center fiber optics hardware.
“Low-latency requirements influence the design and location of data centers so that latency issues can be overcome,” Berman told The Forecast.
The impact of AI on regional data centers is multifaceted, influencing hardware, infrastructure, security and operational considerations, Berman added.
Although the initial promise of cloud computing was all about centralizing things in one big data center, the decentralized AI edge is becoming increasingly important due to the volume of data that’s being generated, according to Alok Shankar, senior engineering manager at Adobe. Because AI must make important decisions quickly, he noted, compute must be located where the data is.
“If you want to make a fast decision — if you want to train your model faster — you infer at the edge,” Shankar said in an interview with The Forecast.
AI Solutions to an AI Problem
Even with all the innovations going on within data centers, energy capacity is a key constraint, according to Skurla.
“As network bandwidth continues to grow, power capacity is emerging as the primary concern for data center expansion — driving a re-thinking of everything from how and, importantly, where data centers should be located,” Skurla said, noting that data centers already account for 2.5% of U.S. electricity consumption and are expected to reach 7.5% by 2030.
That growth in energy consumption means that data centers hosting AI applications require advanced tools to manage spikes in power demand and improve latency.
“Increase in operational load can translate into megawatts of power being drawn from the grid,” Skurla continued.
Ultimately, the technology that’s driving AI data center growth and evolution is the same technology that will help address challenges around power, sustainability and reliability.
For example, Skurla said more data centers need advanced IoT platforms that provide real-time views that help them shift workloads and balance energy consumption without overloading the power grid. Shankar, on the other hand, predicts that AI will enable self-healing architectures that can detect and fix failures and security vulnerabilities while automating software updates and data replication from one data center to another.
“Data center management itself is going to be using AI quite a bit,” concluded Shankar.
Gary Hilson has more than 20 years of experience writing about B2B enterprise technology and the issues affecting IT decisions makers. His work has appeared in many industry publications, including EE Times, Fierce Electronics, Embedded.com, Network Computing, EBN Online, Computing Canada, Channel Daily News and Course Compare.
© 2025 Nutanix, Inc. All rights reserved. For additional information and important legal disclaimers, please go here.