Generative AI and Its Impact on Data Center Design

Data center developers and their design teams will face challenges in obtaining available power and will need to explore novel ways to attain, produce, and deliver that power to the IT equipment (ITE). According to Jonathan Gilmour, Data Scientist, Harvard T.H. Chan School of Public Health Fellow, “Data centers are fixed infrastructure. Decisions today about data center siting and technology will determine the landscape of data center energy demand for years, perhaps decades, as the companies building data centers seek to recoup their investments.”

Growth of AI Computing

IT systems and equipment designed for Artificial Intelligence (AI) computing are not new. But the current power demand for AI systems, and projections for the next decade, far exceed those of the last decade:

• AI deployments on research and specialized cloud applications were put into use by the late 2000s.
• The first major growth in AI expansion came in the early 2010s when advancements in machine learning propelled the demand for AI-specific hardware systems.
• In the mid-2010s, high-powered accelerators such as graphics processor units (GPUs), were beginning to be increasingly standard in hyperscale and enterprise data centers to support diverse AI workloads.
• The late 2010s and early 2020s saw a remarkable growth in AI computing systems.

According to the International Energy Agency (IEA), global data center power use will double by 2030, with AI systems accounting for 40–60% of that increase.

Evolution of Data Center Cooling

Using direct liquid cooling for computers is not a new design strategy. One of the earliest applications for cooling computers with liquid dates to the 1950s for the Defense Department mainframes. The use of liquid cooling for computers continued to grow into the 1980s for enterprise mainframe computers.

By the early 2000s, it was becoming impractical to cool high-performance, super-computing, and enterprise data centers relying on air cooling alone. Based on this challenge, manufacturers began developing new, stand-alone liquid cooling equipment specifically to augment existing air-cooling systems. Equipment such as rear door heat exchangers (RDHX), in-row fan coil units, and overhead fan coil units were used extensively in high-density data centers.

While these systems use chilled water from the facility’s chillers to cool “hotspots” in an air-cooled data center, there are few similarities to equipment and systems used for direct to chip cooling, which is an integrated cooling strategy solely used for the AI cabinets.

AI System Power Densities

Hand in hand with the rapid growth of AI power demand comes unprecedented power and cooling requirements for data center ecosystems that support AI computing. While power and cooling designs such as liquid cooling are not new, current and projected AI power demand is two to three orders of magnitude greater than previous designs. In fact, hardware accelerator manufacturers estimate that power requirements per cabinet (system) will reach 2000 kW by 2030. (Compare this to current high-density air-cooled IT cabinets that will peak at 20 kW to 30 kW).

Fortunately, members of the data center design community have developed concepts including recommendations on parameters such as liquid temperatures and quality, and system resilience. While many of the guidance documents are still in-process, industry organizations, data center engineers and operators, and equipment manufacturers are working to release additional design approaches.

The DLC solutions typically incorporate a cooling distribution unit (CDU) which is essentially a heat exchanger and pump package interfacing the facility water system (FWS) with the technology cooling system (TCS). The cooling liquid is piped directly to a heat sink on the GPU chip using a cold plate, mounted tightly to the GPU die. Hardware accelerator manufacturers are designing AI systems using 100% liquid cooling. (Air-cooling will still be required for other ITE that is not liquid cooled).

Based on Book 1 of the ASHRAE datacom series, Thermal Guidelines for Data Processing Environments, GPU manufacturers’ reference design is based on the W45 class (113ºF) for FWS and the S45 class (113ºF) for TCS. These water temperatures are substantially more than legacy data centers using water as a cooling source.

Data Center Environmental Sustainability

In addition to data center architects and engineers developing novel designs to support massive AI workloads, there are broader, interrelated challenges that transcend the physical limits of the data center facility – energy efficiency, water consumption, and GHG emissions. For example, using DLC will have side benefits beyond simply cooling the AI systems. These benefits will result in enhanced environmental sustainability of data centers:

The high density of AI systems allows for a smaller data center footprint when compared to air cooling. Studies show that the reduced footprint will yield up to 50% reduction in embodied and operational carbon emissions.

Using liquid cooling requires less energy (4x less) to store and absorb heat compared to air. Also heat transfers through water about 25 times more easily than it does through air. Industry studies estimate up to a 22% savings in energy consumption using DJC.

Using the reference design approach for FWS and TCS liquid temperatures will require only a small percentage of mechanical cooling (compressors). This is achieved by using an economizer cycle for air-cooled and water-cooled chillers. (The 35 MW world’s fastest supercomputer, located at Lawrence Livermore National Labs (LLNL), does not use any mechanical cooling.

Employing DLC and other design approaches that leverage the unique operating parameters of the AI systems, will reduce energy and water consumption of the data center. Strategic site selection of the data center taking advantage of mild outside air temperatures, as well as the source fuel mix and water use, will play a role in reducing GHG emissions.

Hurdles for Securing Data Center Power

Allison Silverstein, a former senior advisor to the chairman of the US Federal Energy Regulatory Commission, has voiced concern that the operating behavior of data centers could cause cascading power outages for an entire region. As data centers continue to expand in number and in megawatts, concerns have arisen that utilities and grid operators may have to limit growth, particularly in regions where electricity demand for data centers is already at a peak. While this may be true in some regions, it is not occurring uniformly at the national level.

To improve and expand stakeholder engagement to address these issues, the Federal Energy Regulatory Commission (FERC), the North American Electric Reliability Corporation (NERC), and regional grid operators are holding technical conferences, creating working groups, and drafting policy reform measures that address reliability and fair cost allocation as new large loads reshape the system. However, planning and analysis can result in a trade-off: While data center owners request more power, regulators add contractual requirements to limit financial and operational risks, including penalties for overestimating or delaying power ramp-up plan.

Additional Contributing Authorship by Rajat Wadhwa

Rajat Wadhwa is a passionate engineer with a focus on sustainable and ecologically responsible building design. Rajat is focused on the analysis of energy usage, bio-climatic analysis and application of passive systems wherever possible in order to enhance building efficiency.