Preparing for the petascale: real science on up to tens of thousands of cores

Abstract:

We describe computational science research that utilizes petascale resources to achieve scientific results at unprecedented scales and resolution. The applications span a wide range of domains from investigation of fundamental problems in turbulence through computational materials science to biomedical applications in the forefront of HIV/AIDS research and cerebro-vascular haemodynamics. Much of this work has been performed on TeraGrid’s (hemi) petascale resource, Ranger, located at the Texas Advanced Computing Centre (TACC), since February 2008, when it was declared to be the largest computing platform in the world available for open scientific research. Ranger’s theoretical peak performance of 0.58 petaflops is comparable to the combined peak of all the other resources, available on the US TeraGrid and is almost ten times greater than the latest Cray XT4 supercomputer in the UK named HECToR (59 Tflops).

We have sought to optimally utilize this enormous petascale computer across application domains and scales, exploiting the excellent parallel scaling performance found on up to at least 32,768 cores for certain of our codes in the so-called ‘capability computing’ category as well as high throughput intermediate scale jobs for ensemble simulations in the 128 - 512 core range. We also discuss the parallel performance of some of our codes on the IBM Blue Gene/P system at the Argonne Leadership Computing Facility (ALCF) which has a theoretical peak of 0.56 petaflops. Our work is greatly facilitated by the the grid infrastructure which links these resources to those in the UK and Europe.