Students smash competitive clustering LINPACK world record

The kids are all right

HPC Blog Enormous happenings at the ISC17 Student Cluster Competition, where students from the Friedrich-Alexander-Universitat (FAU) broke the student cluster competition world record for HPL (LINPACK).

This gives the home court German team the coveted Highest LINPACK award.

This marks the first time a German team has won a major performance award on their home soil and is likely to result in the declaration of a national holiday featuring parades and a statue raising in their home town of Nuremburg.

Lins Packed to the Max

On the LINPACK benchmark, it was a pretty close grouping between the top five finishers, with FAU grabbing the top slot with a score less than 10 per cent better than second place Purdue/NEU.

Both teams were sporting GPU-heavy systems. Purdue/NEU had two nodes that hosted an eye-popping 16 NVIDIA P100 GPUs, while FAU took a conservative approach with just using 12 P100s in their dual node box. (Yeah, 12 P100s is conservative….lol)

Pre-competition, the FAU team confided to me that they had tested the 16 GPU configuration and found that their results were actually lower than using fewer GPUs. They speculated that 16 GPUs was simply too much for the PCI bus to handle and that the resulting congestion on the bus was what was slowing their results.

Purdue/NEU didn’t have much time with their machine, so they didn’t have the opportunity to test all of the possible configurations, which is probably why they missed out on the FAU discovery.

EPCC was right in there with their "rule of three" cluster – three nodes, nine GPUs, plus liquid cooling. I would have liked to see them take advantage of their liquid cooling by overclocking their GPUs. I think that would have made the difference between their third place finish and grabbing the LINPACK trophy. But what the hell do I know? I’ve never built a cluster.

Every one of the top four teams beat the existing HPL record of 30.71, established at the ASC17 competition in Wuxi. Nanyang just barely finished below that mark.

Historically, here’s how the LINPACK numbers look in context:

As you can see, scores took a huge leap at SC16 when NVIDIA P100 GPUs came onto the scene. We saw another small improvement at ASC17, but then another largish jump at ISC17.

I’m not sure why we saw such a big bump this time around, but I think it has something to do with the form factors that the winning teams were using. Eight GPUs on a single node allows the GPUs to communicate much more effectively with each other and the host processors vs. having eight GPUs on four different nodes.

The students at ISC17 have also broken the student HPCG record, so stay tuned…