Immersion Cooling
What is immersion cooling
With the growing demand for high-performance computing environments and as computer components become more performant, the datcentre industry is facing increasing power demands and sustainability challenges.
Immersion cooling is an innovative cooling solution that can significantly impact the industry. By immersing servers and other hardware in a specialized cool, non-conductive liquid, immersion cooling creates important gains in power efficiency and carbon footprint reduction compared to traditional air-based cooling methods.
Immersion cooling is particularly effective in managing the heat generated by high-performance processors and GPUs, allowing for better thermal management in densely packed computing systems.
Immersion cooling offers a solution that presents the best possible performance, energy reduction, density and flexibility for datacentre workloads. The technology itself is an IT cooling practice by which complete servers are immersed in a dielectric, electrically non-conductive fluid that has significantly higher thermal conductivity than air. Heat is removed from the system by circulating liquid indirect contact with hot components, then through water-cooled heat exchangers.
Power Usage Effectiveness (PUE) has been adopted as a major KPI for datacentres. The formula is simple: total energy footprint of the facility / the energy consumed by IT.
Air cooling (the method traditionally used in datacentres) relies on fans and air circulation to dissipate heat from the IT components, while immersion cooling uses a fluid to directly cool the hardware, providing more efficient heat transfer. When the average global PUE is 1.55 and immersion cooling can reduce that to below 1.1, there’s no better efficiency approach than immersion cooling in today’s market.
Immersion cooling offers high thermal dissipation and can recover all of the heat created. It’s also clean and environmentally friendly. Removing the fans means that all the energy invested in the server is put to work, which adds value to the data, saving over 10% of IT power and essentially removes a failure point.
Decrease in datacentre CAPEX and OPEX. Aircooling requires large amounts of energy to continuously circulate and cool the air, not to mention run the compressors in refrigeration air chillers within the datacentre. Immersion cooling reduces energy consumption by leveraging the high heat capacity of the cooling liquid, resulting in significant energy savings.
Immersion cooling allows for a much higher computing density compared to traditional air-cooling methods. Since the dielectric liquid can dissipate heat more efficiently than air, tanks can be packed more closely together than racks in a smaller space, increasing the amount of compute housed in that area.
On an IT level, immersing the servers reduces the risk of overheating, thermal stress and oxidation on server components by displacing air from the server components, leading to fewer hardware failures and longer equipment lifetimes. It’s also worth noting that the Asperitas natural convection system is able to function very well on high operating temperatures, high utilisation while being self-managed at the same time. To tackle redundancy, continuous cooling and maintainability can happen concurrently – and this all happens alongside autonomous safety and comprehensive monitoring.
Previously, datacentre operators may have seen immersion as being a tricky choice due to these concerns about maintenance, but with offerings that include expert commissioning and servicing as well as innovative add-on solutions such as service trolleys, maintaining an immersion system is simple and efficient.
An understandable misconception of immersion cooling is that it makes a splash. Right on your datacentre floor. Thankfully, that doesn’t need to be the case. With the right training, PPE and service tools, the reality of working with immersed IT can be quite straight forward.
Single-phase immersion cooling, as the name suggests, is where the coolant stays in a single (liquid) phase, and does not evaporate. The coolant captures the heat from the immersed components and is circulated through a heat exchanger that transfers the heat to a water loop, which in turn can be cooled with an evaporative/adiabatic cooling tower, dry-cooler (radiator), or an existing chiller plant (not required, but compatible).
Two-phase immersion cooling is where the coolant is designed to evaporate (change phase from liquid to gas) at lower temperatures, boiling off when it comes in contact with hot components. The evaporated coolant vapor is then condensed back to the liquid state through the use of condenser coils, typically located at the top of a sealed rack.
Single-phase immersion requires circulation of the dielectric liquids by pumps or by natural convection flow through integrated or external heat exchangers.
Hydrocarbon fluids used for single-phase immersion cooling are typically synthetic fluids, which should not be confused with mineral oil. Mineral oils can often have imperfections, impurities, and limit material compatibility. Synthetic fluids, on the other hand, are manufactured at a molecular level which results in an inherently stable product that offers superior performance and material compatibility.
They are clear, odorless, non-toxic fluids that are readily used in domestic products such as cosmetics and other household products, which have well-established material handling practices, minimal regulations, and no direct GWP (Global Warming Potential). The latter can be a challenge with fluorocarbons.
Asperitas exclusively works with superior synthetic immersion fluids which are well developed by reputable partners and fully tested and optimized for immersion cooling.
One of the advantages of using single-phase Immersion Cooling technology is that the fluids being used are not influencing the business case strongly, while, at the same time, offering the most reliable environment for electronic components.
Unlike fluorocarbon fluids, the hydrocarbon fluid used by Asperitas does not evaporate under normal operating conditions and does not need to be topped up. Regular lab tests are performed on the coolants to test for any variance in critical properties over time. Commercial deployments dating back to as early as 2010 have been running efficiently and reliably, with no degradation in the coolant’s performance.
The fluid used by Asperitas has a very low flammability rate and does not readily ignite. The US National Fire Prevention Association (NFPA) 704 diamond rates such coolants as 0-1-0 substances. This means that it poses no health hazard, has a high flash point, and is stable even under fire exposure conditions.
One issue that can arise when pushing the boundaries of growing industries is the lack of standards and certifications for immersion solutions and deployments. Developing such standards is crucial for industry growth.
The Open compute project (OCP) was initiated by Facebook in 2011 with a mission to apply the benefits of open source and open collaboration to hardware and rapidly increase the pace of innovation in, near and around the datacentre’s networking equipment, general purpose and GPU servers, storage devices and appliances, and scalable rack designs.
Being a prominent OCP member has allowed Asperitas to be a main author and driving force behind several papers such as Design Guidelines for Immersion Cooled IT Equipment and Immersion Requirements. These documents form the first and only industry references for IT equipment OEMs to build their portfolios for immersion, and their primary role is to generate and qualify requirements for immersion solutions. The aim is to foster a landscape of accurate, fact-based technology positioning.
To ensure access to compatible immersion cooling hardware, Asperitas has partnered with a wide range of industry leaders. Each of our partners is contributing to the success of Immersed Computing®.
We are very grateful for the contributions and expertise that Asperitas brings to our community. Their collaboration style and leadership by example is what has made the Immersion Sub-project one of the fastest growing communities in our ecosystem. Asperitas is helping OCP think about solutions for adopting immersion in the datacentre facilities and beyond.”
While optical networking is possible with immersion, it does require some specialized components to work reliably. Some immersion enthusiasts and newer providers claim that you can use any standard fiber optic cable/connector in an immersion environment. However, we have found that to not be factual and/or a reliable way to deal with fiber optics in immersion.
Optical networking is based on the refraction of light through mediums of varying refractive indices (air, fiber cables, connectors, etc.) when a liquid fills a space that was designed to be an air gap, it changes the refractive index and can cause signal loss. In a static environment this could be resolved by optimizing the refractive index of the liquid. In a dynamic environment, however, such as an Immersion Cooling rack, the flow of the liquid, the variation in its density (due to temperature variation) that causes a change in the refractive index of the coolant, and the possibility of bubbles getting trapped in the miniscule gaps can all cause issues with signal reliability.
In addition, materials used in the optical transceivers may not be suitable for operation in immersion fluids. This may result in discoloration, which in turn has an effect on the light signal which travels through these materials. Hence we recommend using specialized optical cables and connectors that deliver a much more reliable connection and signal stability.
From a user perspective you are making a tremendous investment in IT hardware, which should be taken seriously and treated with care. Standard hardware that was originally designed for air cooling can be optimized for immersion. This consists of removing components such as fans that have no use in immersion, and replacing materials such as thermal paste with materials that have better performance and longevity in fluids. Asperitas is working closely with leading OEMs to provide immersion ready servers that may be optimized for immersion or specifically designed to exploit the higher thermal capacity and improved reliability offered by immersion.
Yes, you can. Asperitas has a track record in delivery a range of systems including all components.
Several OEMs offer standard warranty and Asperitas is working closely with system integrators and OEMs to offer you warranty models meeting your expectations.
Yes, you can. Components can be replaced, Asperitas is offering different approaches to do so. The horizontal, open rack design allows easy access to each immersed server individually.
Immersion cooling solutions allows the datacentre to go to places potentially inaccessible before. Whether it is an expansion of the existing datacentre space, inside an office building or closer to the edge of the network like a rural area. This opens up possibilities to integrate IT into other industrial processes to address the global need for energy efficiency.
A single water loop can be shared across multiple liquid cooling modules and the modules can be placed back-to-back and side-to-side. Since there is no air required for the system to operate, large rows of interconnected systems can be placed in relatively small spaces.
Each system requires only power, access to a water loop and data connectivity to operate in a datacentre. Combined with its silent workings, these limited requirements enable high flexibility in deployment sites and scenarios for immersion cooling, inside and outside the datacentre.
The physical location of the datacentre becomes less challenging with the AIC24. Since there is no noise, datacentres can be built closer to the edge.
Yes, you can. Generally though, immersion cooling requires a water circuit running to the systems, which may be more than what you are used to.
Usually, colocation providers will be happy to facilitate immersion cooling, it will allow them to facilitate high density environments without having to make large investments. Asperitas is working with several colocation providers as partners.
Liquid immersion cooling systems such as those offered by Asperitas are designed to be installed on raised floors and concrete slabs alike. And while it may seem obvious, it is important to point out that when talking about floor loading, pressure on the floor is a more important consideration than gross weight alone. Because our racks are horizontal, they help evenly spread weight across their footprint, yielding a floor loading that is often less than that of an air-cooled rack.
Actually, immersion cooling eliminates the need for raised floors entirely, but if you already have a raised-floor infrastructure there is no need to remove it, and likely no need to reinforce or modify it in any way.
While it is true that an individual horizontal rack has a slightly larger footprint than a vertical rack, when you look at the fully burdened datacentre footprint, immersion cooling produces significant space savings: no hot/cold aisles or high ceilings are required, neither is room for air circulation. Immersion Cooling does not need the height you will see in traditional datacenters, in fact in most cases the same data hall can be designed to have two levels with immersion cooling instead of just one traditional level. With immersion you will achieve a higher performance and more IT power per unit area.
Asperitas systems are compliant to the most common system certifications, check your specific provider for details or for tailored certifications for your application. We have track records in serving customers and delivering to datacenters with the highest requirements possible.
For new users of immersion cooling, servicing and maintaining the system is simply different to air-cooled systems. It is not more complicated. Our perspective is that the right approach is to train staff and we offer this service.
For new users of immersion cooling, servicing and maintaining Immersion Cooling is simply different to air-cooled systems. It is not more complicated. Our perspective is that the right approach is to train staff and we offer this service.
All starts with planning and road mapping. Setting sustainability objectives is one thing, but it’s important to also identify your performance requirements for the next few years. Lastly, where do you see your workloads being facilitated? The next step is sharing your project brief. Then it's a matter of gaining experience via remote testing and on-site experiences, starting with piloting and then building your scale-out plan.
Book your free consultation now, we'd love to guide you along the way to a more sustainable and efficient datacentre..
Realistically, a hybrid form of cooling for datacentres is the likely scenario moving forward. At the moment, we’re seeing a variety of use cases – standalone, integrated, owned and colocation. Immersion can be applied in all of these options while keeping in line with regulations such as the European Energy Efficiency Directive. In the long term, we see greenfield data centres as the way to go to fully see the impact on sustainability.”
