Hot Topics:

NCAR's new supercomputer set to run models by summer's end

Yellowstone now being installed in Cheyenne home

By Laura Snider Camera Staff Writer

Posted:
06/09/2012 05:00:00 PM MDT

IBM engineer Jared Prim adjusts cabinets containing pieces of the new National Center for Atmospheric Research supercomputer Thursday at the NCAR facility in Wyoming.
(Michael Smith/Wyoming Tribune Eagle)

All this month, the nuts and bolts of Yellowstone -- the Boulder-based National Center for Atmospheric Research's new supercomputer -- are arriving at its new home in Cheyenne, Wyo.

By July, all of Yellowstone's cables, disk drives, processors and racks should be installed, connected and ready to be tested, first by IBM to make sure everything is integrated and working, and then by NCAR to make sure science models will run smoothly.

By the end of the summer, Yellowstone -- which will be able to process nearly 30 times the workload now being churned through by Bluefire, NCAR's current supercomputer housed at its Mesa Lab in Boulder -- will be ready for researchers, who are already dreaming of the improvements they will be able to make to their modeling capabilities.

Scientists at NCAR and other partner institutions have already put together proposals for how they'd like to use Yellowstone that, taken together, are in excess of its capacity.

"Even with a 30-fold jump over Bluefire, it's already being over-requested," said Anke Kamrath, director of operations and services for NCAR's Computational and Information Systems Laboratory. "That's how pent-up the demand is."

A history of big machines

A supercomputing pioneer, NCAR has had machines designated as supercomputers since 1966. But its first truly massive supercomputer -- which for the first time required its own underground room -- was the Cray-1A, which arrived in Boulder in 1977.

Advertisement

The $8.9 million Cray-1A boasted the ability to perform 80 million computations per second. By contrast, Yellowstone will be able to perform about 1.6 quadrillion computations per second -- 20 million times the amount.

But computational speed isn't the only thing that makes Yellowstone a dream for scientific modelers. All those computations are only as useful as the data they produce -- and for that matter, the data they feed on -- so it was important to design Yellowstone with plenty of memory and bandwidth, so the data can be stored and moved around as necessary.

"If you can't get the data into the computer or out of the computer, that's a bottleneck, and the computer is only as fast as the bottleneck is," Kamrath said. "We've tried to design the system to really be balanced and support data-intensive supercomputing."

Yellowstone is being housed in a $70 million, 153,000-square-foot facility in Cheyenne. About 20 staffers will be onsite at the center, but the researchers who use the system will be able to access the computer remotely.

When Yellowstone comes online, Bluefire will be shut down, largely because it will be cost-prohibitive -- given the amount of electricity supercomputers use and the cost of maintenance -- to continue to operate it. (Yellowstone and the facility that houses it, which is built to high energy-efficiency standards, is expected to consume between 3 and 4 megawatts of power when it's up and running.)

The space now occupied by Bluefire will be used to consolidate many of the smaller computer systems housed at NCAR, which will allow the organization to more easily make energy-efficiency upgrades in Boulder, Kamrath said.

Longer, faster, more complex

All the computing muscle now being built in Wyoming offers tantalizing opportunities for Boulder researchers who depend on computer models to help them more accurately predict weather -- both on Earth and in space -- and better understand how the climate is changing.

In general, more computational power allows researchers to run models over longer time periods, at higher resolution, with greater complexity or more quickly.

In the case of predicting space weather, NCAR scientists may use Yellowstone to run more standard model simulations in a shorter amount of time.

Now, when massive solar flares, or "coronal mass ejections," erupt from the surface of the sun, it's difficult for space weather forecasters to know how large of a geomagnetic storm may result on Earth. In part, that's because scientists aren't able to tell what the orientation of the magnetic field inside the coronal mass ejection is until just minutes before the flare hits Earth. (How the flare's magnetic field aligns with Earth's own magnetic field has a huge impact on the magnitude of the resulting storm.)

"If we can't get the magnetic field direction exactly right (when it leaves the sun) -- and we don't understand all of the physics of that -- we could say, 'What were the ensemble of possible configurations that could happen?'" said Michael Wiltberger, an NCAR scientist who works in the High Altitude Observatory. "Then we can run a whole bunch of simulations with those inputs and be able to provide a probabilistic forecast."

In order to run the number of scenarios needed to create that forecast -- which would outline the probability of the storm reaching a certain magnitude -- the researchers need a computer system that can quickly run through multiple scenarios.

But space weather forecasters may also use Yellowstone to increase their model's resolution, allowing them to make more regional predictions instead of simply predicting the storm magnitude on a planetary scale, Wiltberger said. And Yellowstone could help researchers crack some of the basic physics questions related to space weather, including what causes a flare to actually erupt.

"There's a bunch of people working on these questions, and they need really big computers to be able to do that," Wiltberger said.

Resolving the mountains

Many NCAR climate scientists are interested in Yellowstone's ability to help researchers increase the resolution of their models.

Now, NCAR's climate models can't tease out what may happen on a scale that's less than 100 kilometers, and that means that the models have a difficult time showing the formation of smaller-but-intense "extreme events," including droughts and hurricanes, said Richard Neale, a scientist in NCAR's Climate and Global Dynamics division.

"Things that happen below 100 kilometers -- we have to sort of estimate rather than resolve," he said.

But the impact of a changing climate on extreme events, which can be costly for governments and private entities, is one of the areas where more information is in the highest demand.

Locally, the new computer should help modelers better resolve the Rocky Mountains, and therefore, better understand both how the mountains will be impacted by climate change and how the mountains themselves impact local weather.

On the weather side, Yellowstone may also help researchers input more of the current weather situation -- whether or not thunderstorms are pounding a particular area, for example -- into the model before it runs its next simulation.

"Ten years ago, we didn't know how to do that," said NCAR scientist Bill Skamarock. "If we wanted to start a prediction right now, and there's a thunderstorm out there right now -- a big one -- we did not know how to put that in the initial state of the model."

Yellowstone may also help researchers chip away at some basic science mysteries in meteorology -- such as why some thunderstorms spawn tornadoes and others don't. And similar to space weather prediction, the new supercomputer may also allow meteorologists to run a larger ensemble of possible outcomes for a particular period of time -- tweaking some of the input variables that scientists are uncertain about -- and allowing them to give better probabilistic forecasts.

"If we run all these forecasts forward in time, and then we see how they differ, how they diverge, that tells us a lot about how much confidence we should have in different aspects of our forecasts later in time," Skamarock said.

Boulder is pretty good at producing rock bands, and by "rock," we mean the in-your-face, guitar-heavy, leather-clad variety — you know, the good kind. For a prime example, look no farther than BANDITS. Full Story