Russian water-cooled supercomputers to appear in 2013

T-Platforms has been developing a water-cooled HPC blade-system. The company says the project will take 2 years and $10 million to be finished.

Chief system architect of the T-Platforms HPC vendor, Andrey Slepukhin, told CNews the company was going to develop a prototype of a water-only cooled compute system by the end of 2012. Intended for petascale installations, the product is to be delivered to the market in 2013.

T-Platforms intends to create a system based on CPU or hybrid CPU + GPU architecture. In the first case, the peak performance of the computer would reach 85-105 Tflops and in the second - up to 500 Tflops.

The company is currently working on the system’s concept of architecture. Slepukhin says there is a great number of technicalities to be settled by the developers, for instance, to develop a strong liquid-cooled power supply system, determine the use of optical fiber, and ensure maximum safety for water cooling.

T-Platforms will thoroughly consider each of the above mentioned technicalities regarding their cost, because the advantageous technological solutions should also guarantee a competitive price.

The company currently produces blade servers for 5U and 7U chassis. Unlike these servers, the water-cooled platforms will be a rack-mount solution; the blade modules will be installed directly into rack cabinets. The company will have to design a rack different from the ones currently available on the market , as the latter do not meet infrastructure requirements for the cooling system.

Suffice it to mention that T-Platforms has already equipped its current blade platforms with water-cooling. In particular, it has a solution with a mixed cooling system, that is, water-cooled compute nodes and air-cooled head node, power source, and switches.

The cooling plate in a current system. The Computational node bears against the chassis

The water-cooling here works as follows: cold plates connected through water tubes have inner water channels and are installed into the chassis. The plates receive liquid through a single inlet and the computational nodes imbedded into the chassis bear against the cold plates. There is only one outgate for the outflow of water. Water-tube connectors for liquid inflow and outflow are located beyond the chassis. The developers believe that the placing tube connections outside chassis reduces the risk of water leak and increases the system’s durability.

“This is an interim solution and we are working on a number of technologies to be applied in the next generation of water-cooled supercomputers”, T-Platforms marketing director Alex Komkov says. “However, we will have to considerably modify the water cooling system to not only cool computation nodes with water, but also all other heat-generating components”.

There will be a similar cooling system in the next-generation solution, Komkov says. The water will flow to interconnected cooling plates and withdraw from the whole rack at once, not from each compute node.

HPS systems based on water-cooled platforms have already been delivered in Russia. For instance, a supercomputer of this kind with a peak performance of 117.6 Tflops is installed in South Ural State University. It uses the SKIF-Aurora platform developed in an alliance with the Italian Eurotech company, RCS SKIF and PSI RAS with support from Intel. Its development began in 2008 and the major hardware components, such as the plates, the water-cooling system and the chassis had all been there before cooperation with Russian partners began. There, water is flown to each computational node and the chassis.

Head of the Programming systems Institute at the Russian Academy of Sciences Sergey Abramov told CNews that the connection of water tubes within the chassis was not a shortcoming of SKIF Avrora and there was no need for enhancement because the platform has all the relevant certificates and meets all the safety requirements. One has to see the T-Platforms cooling system in action to be able to assess it, he says.

T-Platforms marketing director says the company will need a $10 million investment to develop a new generation blade system with water cooling. It will use some of its own money and seek support from Vneshekonombank, as well as place bonds on MICEX.

The company has recently decided to issue bonds denominated 100 million rubles ($3,5 million) and in October 2010 Vneshekonombank decided to enter the T-Platforms capital. The deal is still in the making, reports say.

The company does not intend to give up manufacturing blade servers with air-cooling. T-Platforms says it is still going to use air-cooled systems in at least next 2 generations of platforms. The systems will be used for medium and high capacity HPC installations.

Water-cooled platforms for supercomputers have already been developed by world leading vendors, like Cray and IBM. Personal supercomputers with mixed air+water cooling have been developed by the Federal Nuclear Center in Sarov, Russia.