2022-08-31
September 11, 2023
|
blog
GIGABYTE and Asperitas make it easy to run datacentres cool
Writing credit: René Raaijmakers, Editor in Chief at Bits&Chips
GIGABYTE Technology has joined forces with Asperitas to offer immersion cooling solutions that deliver ultimate sustainable performance for datacentres. The collaboration between the rising star in high-end servers and Dutch cooling technology specialists promises to drastically reduce energy consumption and improve power usage effectiveness in datacentres.
Computing power and data storage capacity are still its number one priority, but it's remarkable how Taiwan's GIGABYTE Technology puts power conservation and energy use at the center of its product pitches these days. Not for nothing. Immersion cooling can radically reduce operating costs of its servers and bring considerable environmental benefits for its customers.Research shows that immersion liquid cooling is the most promising technology from a performance perspective[1]. At the same time, it’s clean and environmentally friendly: the heat transfer from the compute to the cooling liquid uses natural convection without consuming any extra energy - a unique feature of the Asperitas tank.“Market demand is what drives our products and performance still plays the most important role,” says Liam Quinn, Marketing Specialist at GIGABYTE. However, in recent years, GIGABYTE has seen that developers of cooling systems attracted the attention of cloud providers with technology that highly increased the power effectiveness of server clusters.
Nowadays customers often approach GIGABYTE directly and they specifically ask them to add immersion to their proposal. “Customers are starting to approach us about immersion cooling at events and so forth,” describes Quinn. “That helps to get more traction.” That’s in short why GIGABYTE chooses to partner with Asperitas and offer an integrated cooling solution to its customers.
Asperitas’ strong presence in Europe was a solid argument for GIGABYTE to partner with the Dutch datacentre cooling experts, with the Taiwanese hardware company identifying the EU as an important market. Quinn: “Our ambition is obviously to grow. We want our name out there in the immersion cooling landscape. When somebody sees the Asperitas product, we want them to also see GIGABYTE.”
However, Quinn doesn’t see the EU as the only front runner in more sustainable datacentres. “We see the same interest in Asia and the US. In Europe, the Asperitas brand name is strong and that helps to build confidence in deploying immersion solutions globally.”
Asperitas started to work with GIGABYTE in 2019 for a project in Europe. The partners worked for over a year on a server optimisation project. “This is in fact a certification process where we look into the feasibility and simulate the server performance in immersion until we get to a final design,” says Maikel Bouricius, Chief Commercial Officer at Asperitas. During this process, an air-cooled server was prepared for immersion in liquid with minimal changes. Bouricius: “The liquid gives great benefits for cooling performance and for the life span of the product. In effect, the fluid is a more stable environment.”In order to immerse a compute platform, simple development work needs to be done to check for chemical compatibility between the designed-for-air components and the dielectric fluid. “This work is done with our experience in immersing hundreds of servers in the liquid over the years,” says Bouricius. “We’re also working with partners like Shell who have a great depth of knowledge on this.”
Asperitas additionally looks into the heat sources and assesses what they need to focus on in an immersion cooled environment. “We make calculations using computation fluid dynamics to assess whether the boards will be cooled sufficiently”, says Bouricius. “If necessary, we propose changes to either the composition or the heat sinks to optimise for immersion.”
“Asperitas and GIGABYTE’s combined expertise results in a winning formula,”, says Andy Young, Chief Technology Officer at Asperitas. “We’ve been sharing our knowledge and have been working together on projects for specific customers. That translated to a general release: GIGABYTE is now launching immersion ready servers equipped to use with our solution.”[1] Source: OMDIA | Data Center Liquid Cooling Market Analysis – 2022
Dell’Oro Group forecasts that immersion cooling will grow at the fastest rate relative to other datacentre thermal management technologies in 2022. Lucas Beran, Principal Analyst at Dell’Oro Group expects that will continue through 2025, driven by sustainability initiatives from cloud service providers and large enterprise. “With semiconductor component and server OEM validation for immersion cooling growing to directly respond to market demand for IT that suits various workloads, these efforts will further drive the scale up to larger deployments.”
The need for computing power places increasing demands on energy management and savings. “Single GPUs on our PCI-cards are hitting a power consumption of 300 watt plus,” says Quinn. “CPUs are going the same direction. So we're dealing with a lot more heat than usual.” Bouricius added: “With that in mind, there are certain variables in a system to optimise: the immersion cooling tank, the fluid and the server. This is where our partnership ecosystem comes in. Asperitas and Shell are closely partnered on the cooling fluid products, and the partnership with GIGABYTE is focused on the server readiness. Partners working together in synergy turns into value for our users.”
As well as the power consumption per chip, server systems deal with an increasing component density. “Servers with two AMD EPYC processors and eight GPUs are now common. Those are demanding at least 3000 watts. That’s a lot of heat that has to be removed,” says Quinn. “But in new server generations we’re going to move to even greater heat dissipation. The current system configurations will simply not be sustainable in the future. If customers want this sort of peak performance in their large datacentres then they have to turn to more efficient means, like using immersion cooling technology.”
Quinn compares the partnership with Asperitas to the way automotive companies work with their suppliers. Car manufacturers like Volkswagen and Mercedes concentrate on core technology like the engine and software. Parts like body, sunroof, doors and exhaust pipes are outsourced in close collaboration. Suppliers like Bosch and Volvo do not simply produce to order, they have their own development roadmaps for car modules and subsystems. The only difference is that datacentres often separately choose their compute platform and cooling systems.For GIGABYTE, a partnership with a provider that can concentrate on immersion cooling technology development and production makes a lot of sense. With a mind-boggling amount of computer platforms and other hardware like power supplies, it has a lot of product lines to develop and maintain. In the field of servers alone, GIGABYTE has seven series of compute platforms. In total there are over 100 products to choose from with the current gen products.
For instance, GIGABYTE’s high performance G series of GPU-centric servers comprise of more than ten families of configurations. Customers can choose from a processor menu of AMD’s EPYC and Ampere’s Altra to Intel’s Xeon. “Our G-series can support heavy graphics and compute workloads,” says Quinn. “We have a lot of different models, segmented for what the market is asking for. At the same time, we do this because it makes us more price competitive. Coupling with immersion cooling technology gives us a reason to offer another variant.”
Motherboards bring in almost a third of GIGABYTE’s Technology revenue stream. “Motherboard manufacturing is the base of the company,” comments Quinn. “At the server division we rely on the strength of our motherboards. This means that we can price things well by making small adjustments. That is a major part of the demand creation.”In March 2021, Gigabyte Technology decided to spin off its server business as a wholly owned subsidiary under the name of Giga Computing Technology. Currently, servers account for 20 percent of Gigabyte Technology's total revenue stream – 4.4 billion dollars in 2021. Graphics cards bring in the biggest market value at 40 percent. Another 10 percent is income from its laptops and other PC components.
GIGABYTE uses a system architecture in which processors, motherboards and cooling systems are integrated with ease. Quinn: “Modularity gives us plenty of room to differentiate. We don't just use x86-chips from Intel and AMD, in our product lines we also use ARM based CPU’s that are currently produced by Ampere Computing. Modification for immersion is one of the options. All these partnerships, with suppliers of microprocessors, graphics engines and immersion providers give us the leverage to be fast to market.”
Datacentres utilise a number of methods to keep components at optimal temperatures. Traditional datacentres opt to cool their IT by moving air in combination with heat exchangers and cooled water. However, air cooling is simply not sufficient to cool down 300 watt plus high-performance CPUs, FPGAs and GPUs.With the introduction of immersion cooling, the air-cooled servers are optimised to make them suitable for contact with fluid. “We add brackets so we can install the systems vertically,” explains Quinn. “In this way all I/O and power connections are plugged into the side facing up. Obviously there's no need for fans because there is passive circulation of the fluid in the tank. At the same time, the baseboard management controller (BMC) can do without communication with the fan and its warning sensors. So it basically comes down to adjustments to make sure it's compatible with the design of immersion cooling tanks.”Specifically high-end complex systems are moving to immersion. Quinn: “I expect that a respectable percentage of our server revenue will come from the immersion portfolio in five years time. You have to realise that when we are talking immersion, we are talking large scale. Customers are not buying immersion for two servers. It typically supports 24U tanks, but they can be easily scaled. It’s definitely targeted at the bigger datacentres, like those in the cloud industry.”
The hunger for power in the industry is really pushing the limits of air cooling. “We’re now seeing servers emerging in the market where you need extra energy consuming components like chillers to reduce the temperature,” says Andy Young. “That’s why you see chip OEMs offering SKUs (stock keeping units) for their processors which are specifically designed for liquid cooling. It's a natural development. We’re excited to see equipment OEMs make design choices for heat sinks and materials to prepare their servers for immersion right from the start.”
To download this article, please click here.
