Post Your Comment

28 Comments

The problem with oil is replacing components, a key element of large installations that might suffer a failure a day. Replacing an oily component means you smell like bad fries afterwards. Or maybe it's another sort of oil. Any info on which hydrocarbon being used and the specific heat capacity? :)Reply

As far as I can tell, they're C4 to C22 alkenes that are polymerised and further hydrogenated to alkanes. Patented chemicals are sometimes the hardest to find info of, and I suspect that these are a range of PAOs rather than one single type.Reply

If the oil is pure alkanes and alkenes, it shouldn't have any smell at all. The aromatics and sulfurous compounds are the ones that have an odor. I work with FT Diesel in the lab, which this basically is, and there is no odor to it either. The petroleum Diesel with sulfur compounds and aromatics is the one that has the odor.

I would think the biggest issue with replacing components is keeping the oil from spreading to everything in the room as components are moved around and replaced.Reply

Yes, the smell was a small bit of comedy :) I would imagine that if any sulphurous (natch, UK spelling ;]) links were present they'd attach pretty quickly to any exposed gold (excess pins etc) due to the low binding barrier therein.

The name PAO is pretty generic, ExxonMobil have several SpectraSyn SKUs, and the datasheets on most of them had them as pretty viscous as far as I could tell. That wouldn't do much in terms of moving the oils around; I'd imagine there's a big-ass pump for each unit so push it through with fans along the way. Reply

The data sheet I saw claimed it was white mineral oil (your dielectric alkenes) with their own patented additive mixture (usually fluorocarbons) and I would assume some antioxidant to prevent free radical breakdown of the alkenes.

They claim there is no evaporation and that the liquid need never be topped off or raplaced. The sheet also said the liquid is pumped through a heat exchanger where it uses tap water (hot or cold) to cool the system.Reply

In large scale installations the minimum size of a swap out module is much larger than in your home computer. I don't have any specific information about super computer installs; but it's probably similar to how huge internet company data centers work. Ex When MS went to containerized data centers their minimum swapout unit was an entire shipping container of computers. Individual computers failing within one were just disabled via admin tools.

Judging by the diagram above, the "swap out module" might not be much larger than a typical blade server, as it appears to be a self contained unit consisting of 2 CPU's, 4 GPU's and associated system board(s).

Also, there's a decent chance that the module itself has no moving parts which are generally the first thing to go on a system. In the facility I work in, we only have a handful of non-disk or fan (or PS, which is usually the fan) related failures a year across a couple thousand servers. Get rid of the moving part failures, and we'd probably be swapping out less than one "module" a month at a location like mine.Reply

Um, just a correction that I would only make on a site like this but oil has nowhere near the heat capacity of water. Besides pure hydrogen or helium only anhydrous ammonia exceeds the heat capacity of water.

I believe they use "white" mineral oil (baby oil, liquid parrafin) with additives of Teflon like chemicals to boost heat capacity a bit. Still the heat capacity is still probably less than 1/3 of water.

Of course there is much more mass so the heat capacity of the whole system is better than if you could somehow use distilled water in a sealed vessel.

My understanding is also that the system is cooled by using tap water and a heat exchanger to answer someone's question.Reply

It may have lower heat capacity than water, but what it is good at is absorbing heat...I would guess that it absorbs heat at a higher rate, much like aluminum absorbs heat faster than steal. With enough volume, it has better cooling efficiency than any closed-loop H2O system.Reply

Got to love AT's eye-poking bias, I've been used to "top 5" or "top 10", but top 6? Now that is a curious choice. I bet it was intended as a top 5 chart, just like the second, but was expanded to 6 just to squeeze one more intel reference inside, god forbid AT put a "top 5" chart where intel is outmatched by ibm chips and has only one intel entry, just as much as amd. For the second chart 5 entries seem to suffice, probably because all systems are intel.

Yeah, keep calling yourself "the most trusted" ... Articles without in-your-face brutally obvious bias at AT are becoming as rare as a hen's teeth. There is not even remote subtlety...Reply

It would have been top 5, but since the only new supercomputer in the larger top 10 was Piz Daint at #6, I stretched it by one so that we could list those numbers and talk about it. Vulcan, SuperMUC, JUQUEEN, and Stampede were all online in June. Or would you rather we ignore the most powerful supercomputer in Europe?Reply

They need to develop a special coolant liquid that is efficient at heat conduction yet has electrical insulation property. Most material that conducts heat does it by conducting electron, so this is super tricky. But if this is available, I can't see why all computer systems won't be submerged cooled from then on.Reply