Data centers are heating up. High performance and AI workloads demand a level of compute density that we haven’t seen before. The power requirements demanded by modern server processors, has more than triple in a decade. Adding to this, accelerators have made great progress in data analytics and AI, but they too add to the power envelope in such a way that the data center just can’t keep up. It’s time to go back to the drawing board to create a new design.

Air Cooling Falls Short

Data centers face increasing challenges from the amount of heat that is generated by components that have a high-power draw, such as CPUs and GPUs. These high temperatures can be detrimental to the overall performance of a system. Systems may have to throttle down to lower clock speeds to maintain availability or huge investments must be made to expand the air cooling system.

Additionally, besides power hungry components, modern high-performance systems have also increased server rack density, causing yet another heat issue. High density servers can house up to eight CPUs in a single 2U server, or 8 GPUs and 2 CPUs in a single 2U server. Now when these servers are stacked upon each other we have further compounded the issue of how to deal with the heat removal. This may also place restrictions on how much power is able to be supplied to a processor without reaching a heat barrier, preventing you from utilizing top tier performance, especially when you also require a large number of drives that can interfere with air flow.

An alternative choice to maximize the use of high performance servers is to use a complex direct-to-chip liquid cooling solution, which comes with a hefty price tag, but can certainly remove the heat more efficiently than air cooling. However, direct-to-chip cooling still does not achieve the efficiency that is possible by immersion cooling and often it still dumps the heat into the data center.

Nowadays we commonly see the average rack power density in a data center approaching 20kW/rack, but HPC applications are going far beyond this, requiring a solution to handle a 30kW/rack, or more, and one that does not rely solely on a facility’s air conditioning. Enter immersion cooling. To transfer heat quickly, immersion cooling uses a dielectric coolant that is an excellent conductor of heat and has a greater heat-carrying capacity compared to air. The direct contact between the coolant and the processors (occasionally heatsinks too) allows for faster heat transfer from the hot components compared to using a heatsink and air flow to pull the heat away. Because of these immersion cooling solutions it is possible to have a self-contained system that can handle up to 100kW/rack, which is a density almost unheard of until now.

Immersion cooling

Liquid cooling has been in use in data center applications for over 30 years. Of the two most common immersion cooling methods, single-phase immersion cooling is the far simpler and quicker to get data centers up and running. In this instance, servers are placed vertically in a tank with an environmentally friendly synthetic fluid, aka coolant, that is an excellent conductor of heat but not electricity. From here the cooling process starts with a circulating loop that uses a coolant distribution unit (CDU) to pump in the cool or slightly warm coolant into the tank’s bath. As the components heat up, the direct contact with the coolant transfers the heat from components to the coolant. This warm coolant is then piped out to the CDU where a pump and heat exchanger remove the heat using a cooling tower, dry cooler, or a chiller and then feed it back into the tank. All in all, the process maintains high availability and efficiency thanks to collaborations with immersion cooling leaders.