Oak Ridge morphing Jaguar supercomputer into 10+ petaflop Titan

The Energy Department is upgrading its Jaguar supercomputer to make it an even more powerful tool for big science, and possibly reclaiming the title of world's fastest supercomputer.

Located at the Oak Ridge Leadership Computing Facility (OLCF) in the Oak Ridge National Laboratory in Tennessee, Jaguar is the most powerful supercomputer in the United States, said Leo Williams, a lab official. Once the upgrades are complete this fall, he said the modified system will be renamed Titan and be capable of 10 to 20 petaflops (a petaflop is a quadrillion floating-point operations per second).

Testing for the first set of upgrades were completed in early February and included evaluating scientific applications for molecular dynamics, high temperature superconductivity, nuclear fusion and combustion. The initial upgrades bumped Jaguar’s processing speed from 2.3 petaflops to 3.3 petaflops.

As a part of the upgrade process, Oak Ridge technicians replaced Jaguar’s AMD Opteron cores with the latest 6200 series and increased their number from 224,256 to 299,008. Two six-core Opteron processors were removed from each of the computer’s 18,688 nodes and replaced with a single 16-core processor, Williams said. Jaguar’s interconnect was also updated and its memory was doubled to 600 terabytes.

As Jaguar is transitioned to Titan, its operating architecture is changing to allow users to optimize their applications to run on the accelerated system. An important part of this is the addition of NVIDIA graphical processing units (GPUs) to 960 of Jaguar’s 18,688 compute nodes. These GPUs are the first part of a larger GPU installation scheduled for later this year.

The GPUs act as accelerators and give scientists a major boost in computing power while making the computer more energy efficient, Williams said. GPUs also provide the parallel processing capability needed for Titan to reach 10 to 20 petaflops in the same space as Jaguar and with the same power requirements. The GPUs are capable of running hundreds of computing jobs simultaneously, while the Opteron processors have 16 cores that can carry out 16 simultaneous tasks, he added.

Another important part of the transition to Titan is keeping the computer accessible to researchers. “During our upgrade, we have kept our users on Jaguar every chance we get,” said Jack Wells, OLCF science director in a statement. “We have already seen the positive impact on applications, for example in computational fluid dynamics, from the doubled memory.”

Jaguar currently stands third on the Top500.org’s list of fastest supercomputers, although its current increased power will likely more it into second, leapfrogging China’s Tianhe-1A system, which last checked in at 2.57 petaflops.

Japan’s K computer still leads the pack, scoring 10.51 petaflops in November, which Titan would surpass if it reaches its projected speeds.