THE LARGEST SIMULATION OF THE UNIVERSE EVER DONE SHOWS THE STRUCTURE OF DARK MATTER

The universe may have begun with a bang, but the images that reach us
from 379,000 years after that singular instant 13 billion years ago
present a fairly mundane picture. The most notable characteristic is
uniformity. Over immense distances, the temperature of the unimaginably
hot matter spread evenly through the early universe fluctuated by mere
thousandths of a degree. Yet those tiny fluctuations generated the
diverse splendor of the galaxies, nebulas, stars and planets we see
today.

Two years ago, a satellite — the Wilkinson Microwave Anisotropy
Probe (WMAP) — captured the first light that escaped from
that hot, uniform early time, providing astronomers with a baby picture
of the universe. With each passing month, sky surveys and x-ray
observatories add more details to fill in the gaps between then and
now. And these observations are only the beginning. In the coming
decade, a new wave of missions promises deeper, sharper views into
early periods of structure formation.

It’s an exciting time for astrophysicists, with one question
uppermost in their minds. How well will the new information match up
with theories about formation and evolution of the universe? If gravity
is the primary force sculpting the heavens, as theories predict, then
what structural features should astronomers expect to find, if they
look in the right places, within the huge forest of emerging data?

Jeremiah Ostriker, Princeton University.

Paul Bode, Princeton University.

Large-scale computational simulations play an indispensable role
bridging the gap between theory and reality in our burgeoning knowledge
of the cosmos. To help narrow that gap, astrophysicists Paul Bode and Jeremiah
Ostriker of Princeton
University used LeMieux to carry out the
largest simulation of the universe to date. Starting with the baby
picture from WMAP, and depicting the universe with unprecedented
detail, they harnessed LeMieux’s parallel-processing power to
evolve the baby cosmos forward to the present.

Unlike most simulations of cosmic structure, which start with a
section of the universe and look only at the end result, Bode and
Ostriker assembled a photo album with which to view the universe as it
grows up. Designed to facilitate comparison with observations, their
album presents the universe as an Earth observer sees it. “We end up
producing a virtual night sky,” says Ostriker, “which anyone can then
study in a computer.” With analysis still underway, they’ve already
turned up hints of some as-yet unconfirmed characteristics of the early
universe.

Computing in the Dark

With big help from LeMieux, Bode and Ostriker populated their
universe with two billion virtual particles — each the size of several
galaxies — twice as much granular detail as the most ambitious similar
simulations. As a concession to computational economy, however, their
simulation takes place in the dark. The virtual universe contains no
flowing gases and igniting stars.

All two billion particles represent dark matter — a mysterious
type of mass we cannot see. These particles, which attract each other,
are also interacting with a still more inscrutable, gravity-less
component that makes up about 73 percent of the energy and mass balance
of the universe, so-called dark energy, which scientists theorize tries
to push space and everything in it apart.

Closeup of a large dark-matter halo, about six million
light years on each side. Brightness corresponds to density.

Tracking the interactions of two billion particles over 13 billion
years to build a virtual model of the universe presents a large
computational challenge. “It’s just at the edge of what you can do,”
says Ostriker, “that’s why you need the biggest supercomputers.”

LeMieux’s combination of number-crunching power and storage
capacity provided the combination needed to compute the position of the
particles and store their arrangement through time. “It’s the whole
package, really,” says Bode, “lots of processors and lots of memory,
lots of disc storage as well.”

Even with these computing resources, however, modeling the
gravitational landscape shaped by two billion dark-matter particles
depends on software ingenuity. Gravity acts over long distances, and
every particle shapes the gravity that acts on every other particle. To
take advantage of parallel processing, particles are parceled out to
different processors, and the need to calculate the force exerted by
the particles at one processor on particles elsewhere can create an
intra-processor traffic jam of messages.

“You have to figure out a way to avoid spending all your time
passing messages around,” says Bode. The solution, first developed by
former Princeton graduate student Guohong Xu, and continually modified
and refined by Bode, splits the force affecting each particle into two
parts, a long-range part that comprises the effect of all particles and
a short-range part that accounts for the gyrations of the particle’s
neighbors.

The software implementing this algorithm, called Tree-Particle-Mesh
, made efficient use (90 percent scaling) of 420 LeMieux processors,
and with five days of computing built the virtual dark-matter universe.

Cold Dark Water in the Valleys

What an observer (in lower-left corner) might see in a 15 x
90-degree wedge of the sky with an x-ray telescope (as
projected on the 2D plane of the page). Each dot is a
cluster containing anywhere from 100 to thousands of
galaxies, color corresponding to mass (increasing from
violet to blue, green, red, yellow).

Much more than cold interstellar dust, black holes, and dark, dead
stars, the exact nature of most dark matter is unknown. Scientists
suspect, however, that dark matter makes up about 24 percent of the
universe’s mass and energy and exerts gravitational force. Luminous
matter contributes only 3 percent, meaning that the gravitational
landscape of the universe is defined largely by dark matter. In the
Cold Dark Matter theory, which Bode and Ostriker implemented on
LeMieux, this means that dozens, hundreds, sometimes even thousands of
galaxies cluster in clumps of dark matter, called halos.

“If we can track all of the dark matter,” says Ostriker, “then we
have a good picture of the structure within which the galaxies find
themselves. We take what we think is the right model of cosmology, we
put in the initial ingredients — which are basically the fluctuations
that have been seen by the WMAP satellite — then we turn the crank on
the computer and allow gravity to act with these little ripples. We
find dark matter accumulating into halos and more massive halos. And
they have substructure and merge and do all sorts of wonderful things.”

LIKE WATER IN A RIVER VALLEY, DARK MATTER POOLS INTO FILAMENTS

Many of the photos from this virtual album will provide key points
for comparison with observations. Because galaxies are packed inside
dark matter and carried along by the speed of the dark-matter halo
surrounding them, for instance, it’s possible to compare with
observational data on galaxy velocity. Ostriker and one of his students
are cataloging the speed of dark-matter clusters from the simulation to
see how this velocity distribution compares with the speed of galaxies
astronomers are cataloging from observations.

This thin-slice snapshot through the simulation volume,
about 3 million light years thick by 4.5 billion light
years on each side, shows the filamental structure of
dark-matter clusters. Brightness corresponds to density.

With a working assumption that galaxy structures are influenced by
the dark matter that envelops them, Bode has tracked the evolving shape
of the largest clusters of dark matter in the simulation. In early
periods of structure formation, Bode found that clusters were more
aligned and elongated than expected, supporting the idea that matter
pooled into strung-out filaments, much as water migrates to and flows
down the center of a river valley. This effect is more striking than
expected, says Bode, and as observations of large galaxy clusters at
earlier times come in, it will be interesting to see how well the
simulation matches up.

Bode is also looking forward to comparing the number of giant
clusters of dark matter in the simulation with the number of galaxy
swarms in the real universe. If the simulation’s mass density — a key
theoretical parameter that describes how closely mass was packed in the
beginning — is larger than in the real universe, the simulation
clusters would come together faster and form larger clusters than in
the universe. If the simulation is off in the other direction, it will
have fewer giant clusters than the real universe.

Bode and Ostriker are also using the gravitational potential of the
dark matter distribution to calculate the temperature of gas within
dark-matter clusters. “It’s an imperfect connection,” says Ostriker,
“but right now it’s the best tool we have. Until now we didn’t even
have that option because we couldn’t make simulations of anything on a
big enough scale to compare to the real universe. We could only do
little pieces.” With dark matter particles and LeMieux, it was possible
to do a much larger section of the universe, one of the largest volumes
of space ever simulated. “These simulations enable you to look all the
way back through space to the beginning.”

What if the simulation doesn’t match up with observations? That’s
the beauty of computational simulations, says Ostriker. They make it
possible to systematically test and adjust theory. “We can then do
another simulation, with a different cosmology. We’ll increase
dark-matter content, or we’ll change the dark-energy content, because
in fact we don’t know these quantities very well.” As observations
become more detailed and simulations more accurate in representing
theory, science will move step-by-step, says Ostriker, toward knowing
what initial features went into creating the universe.