Relying on Blacklight’s shared
memory, a team of astrophysicists is running the most sophisticated, largest simulations yet
undertaken of when the cosmos first began to blaze with islands of light

Before
there was a Milky Way galaxy, a solar
system or planet Earth, the Universe — as if taking a nap after the birth effort of the
Big Bang — was wrapped in a blanket of cosmic fog. There were as yet no stars nor
galaxies. Cosmologically speaking, it was the Dark Ages.

Initially, in that first mysterious microsecond about 13.7 billion years ago,
there was light. And then an instant after the Big Bang, as the prevailing cosmological
theory is often called, matter was an expanding soup of elementary particles, quarks and
gluons and photons, which in turn evolved into a plasma, an ultra-hot swirl of protons,
neutrons and electrons — with temperatures too hot for atoms to form. As the plasma cooled
and the rapidly growing baby Universe was still very young — around 380,000 years, protons
and electrons came together and made neutral hydrogen atoms.

The cosmos had
recombined, and by this time the unimaginably hot initial spark had ballooned into immensity
and cooled to about 3,000° Kelvin.
Besides the dimming echoes of the Big Bang, the cosmic background radiation, nothing was hot
enough to radiate. The Dark Ages passed, obviously. But how did light come back into the
cosmos?

“This is one of major frontiers in astrophysical research,” says Princeton University theoretical physicist and cosmologist
Renyue Cen. “When did the first stars, black holes, galaxies and quasars form? These
questions are fundamentally important.”

Cen and Carnegie
Mellon cosmologist Hy Trac lead a team of physicists — including post-doctoral
researchers Nick Battaglia and Aravind Natarajan and graduate student Paul La Plante —
undertaking a series of very large-scale computational simulations to help answer these
questions. What astrophysicists understand is that, gradually, tiny ripples in matter
started a process by which gas coalesced and ultimately collapsed, under the action of
gravity, to form the first stars, lit up with nuclear fusion, and quasars, powered by black
holes.

“This marks the emergence of the first luminous bodies in the Universe,”
says Trac. Over the next few hundred million years, ultraviolet light from these first stars
and galaxies converted the gas surrounding them into a much hotter, thinner plasma of
protons and electrons — “reionized” it — and the Universe came to look much like it
does today: a sea of blackness dotted with islands of light.

In between
recombination and the start of reionization, a period that, relatively
speaking, didn’t last long — a few hundred million years — things happened by which
the Universe began to structure itself into a vast web — sheets, filaments and knots of
matter. Within this cosmic web, galaxies formed and processes originated that, over billions
of years, led to living organisms and consciousness — which allows scientists in 2012 to
try to understand how, out of inanimate energy and matter, we got to where we are.

Radiation from Neutral Hydrogen This graphic from the simulations
shows 21 centimeter radiation (increasing from blue to red, in thousandths of degrees
Kelvin) emitted by neutral hydrogen in a 5° x 5° map of the sky at a time when the
Universe was approximately 500 million years old. Upcoming radio experiments such as the Low
Frequency Array and the Square Kilometer Array will measure these neutral hydrogen
regions.

“We have basically no
information,” says Cen. “That period of darkness to the end of reionization is a black
box.” To delve into this black box more deeply than has been done until now, Cen and Trac
turned to a sophisticated software approach, RADHYDRO, which they developed for these
studies. RADHYDRO incorporates physics of three mechanisms involved in shaping the cosmos as
it emerged from darkness: gravity, hydrodynamics, and radiation.

Using PSC’s Blacklight,
specifically because of its large amount of shared memory, the researchers have simulated a
larger chunk of the reionizing Universe than previously attempted. Their simulations, still
underway, have begun to zero-in on spectral signatures — the imprint of electrons released
by reionization on the cosmic microwave background radiation and a signal (the “21 centimeter line”) produced by
neutral hydrogen atoms.

More precise information on these signatures of
reionization will help to guide several large-scale observation efforts soon to be up and
running. Some are space-based — the Planck Space Observatory, launched
by the European Space Agency, and NASA’s James Webb
Space Telescope — and others are precision ground-based telescopes. “We are making
maps of the sky,” says Trac, “at various wavelengths and calculating theoretical
predictions to compare with observational data.”

A Three-in-One Model

Even with the most powerful supercomputers, it isn’t
possible to model every atom, proton and photon of light in the entire Universe. The
researchers, nevertheless, have taken their work beyond previous efforts at modeling cosmic
reionization. Their innovative software RADHYDRO is more comprehensive in the physics it
incorporates than prior models, and they are modeling a larger volume of space with higher
resolution in the quantity of particles and light rays they represent within that volume.

“Blacklight makes it possible for us to run the largest
simulations of reionization in the world.”

RADHYDRO includes gravity, which
takes into account the invisible substance that comprises most of the mass in the Universe.
“Eighty-five percent of the Universe is in the form of dark matter,” says Trac, which
interacts with other matter — including the protons, neutrons, and electrons that make up
the visible Universe — only through gravity.

With hydrodynamics, RADHYDRO takes
account of cosmic gases, primarily hydrogen and helium, by tracking their evolution as a
fluid. At this very large scale, says Trac, rather than thinking of gas as individual
hydrogen and helium atoms, it can be effectively treated as an ideal fluid. The microscopic
interactions are accounted for in the fluid equations that describe macroscopic properties,
including extremes in pressure experienced by gases in space.

RADHYDRO’s third
component, radiation, distinguishes this code from most other cosmological modeling, which
generally doesn’t include the physics of electromagnetic radiation. As emitted from normal
matter in space, radiation influences the evolution of the reionizing Universe. “We use
radiative-transfer algorithms,” says Trac, “that follow the propagation of radiation
from early stars and galaxies out into expanding spacetime.”

In volume, the
calculations are almost unimaginably vast. Their most recent simulations cover a cube of 143
million “parsecs” — a little under
500 million light years — on each side. In miles, that’s three followed by 21 zeros.
This compares with a diameter of the Universe at the time of 12 billion parsecs. The
simulation, then, is 1.25 percent the width of the reionizing cosmos. About right, says Cen,
to capture the large-scale graininess of the Universe at that point.

Their
simulations aren’t just the largest yet attempted for this period of the cosmos. They are
also highly detailed. Their virtual box contains eight billion particles of dark matter,
eight billion gas elements, and two billion light rays. “The more particles you have,”
notes Cen, “the more resolving power.”

Traces of the Dark Ages

Princeton cosmologist Jeremiah Ostriker isn’t involved in this project but is a pioneer
in simulating this epoch of the cosmos. The difference in complexity and detail between his
own earlier work (first with Cen and then with Princeton collaborator Weihsueh A. Chiu) and
Cen and Trac’s simulations, he says, “is the difference between modeling traffic with
bumper cars and modeling it with all the detail of a superhighway. Among cosmology models,
this is a real detailed test. When we didn’t have this model and tried to make
predictions, we obtained much less accurate answers.”

The Cosmic Microwave
Background This graphic from the simulations shows temperature fluctuations
in the cosmic microwave background (CMB) radiation generated due to cosmic photons
scattering with fast-moving electrons during the epoch of reionization. This temperature
signal, which registers in millionths of degrees Kelvin, can be positive (red) or negative
(blue) depending on whether the electrons are moving toward or away from us. The square
represents a 15° x 15° map of the sky that spans a time period from when the Universe was
approximately 200 million to one billion years old. Ongoing experiments such as the Atacama
Cosmology Telescope and Planck Space Observatory will measure these temperature distortions
in the CMB.

Although their work is still underway, Cen and Trac and
their colleagues have produced several papers. “For the first few papers in this
series,” says Trac, “we are describing the method and studying how various observable
phenomena change when we alter the reionization process.” This will help theorists
understand data coming in from current and future telescopes such as the Atacama Cosmology
Telescope, Planck Space Observatory, the Low Frequency Array, and the Square Kilometer Array.

A key parameter the simulations track is how photons from the cosmic microwave background
(CMB) — the low-frequency, microwave rumbles of the Big Bang — scatter against free
electrons. By definition, reionization liberates electrons — splitting the neutral
hydrogen of the Dark Ages, making it possible to study, says Trac, “the imprints of
reionization on the CMB temperature and polarization.” Another key parameter is a signal
neutral hydrogen emits at a wavelength of 21 centimeters. The researchers are beginning to
close-in on how observations of this 21 centimeter radiation will constrain how and when
reionization must have occurred.

The researchers are working to scale up their model — to run efficiently on more
processors, for which PSC staff have been crucial. In particular because of RADHYDRO’s
incorporation of radiation physics, Blacklight, with its large shared memory, is the best
possible machine for this work. PSC scientists Roberto Gomez and Rick Costa helped to
overcome obstacles in efficiently using software called OpenMP, which allows the software to
communicate among processors. “Because these photons are always moving,” says Trac,
“communicating them between different processors is very difficult. Blacklight makes it
possible for us to run the largest simulations of reionization in the world.”

Cen and Trac have run RADHYDRO on Blacklight efficiently with as many as 512 processors
and are working to use 2,048. With scaling up the simulation, the researchers plan to
include three times more particles and a larger volume; the 2.048 processor simulation will
include 29 billion dark matter particles, 29 billion gas elements, and 17 billion light
rays. “We’d like to have the ability to resolve small dark matter halos where small
galaxies form and reside,” says Cen, “which were the bulk of the luminous galaxies at
this period.”

With increased resolution, the simulations will, for instance, help to determine what
kind of luminous matter first began radiating — stars or black holes — as the Universe
reionized. Stars and black holes leave different imprints, says Cen, and the simulations can
provide concrete clues that will help to optimize the search strategies of space and
ground-based observation.

Eventually, telescope observations and simulations will feed insights back and forth,
helping the other to advance. By knowing what we’re looking at, we can better understand
how the early Universe worked — and how it became the collection of bright islands of
shining stars and galaxies we see today.