Share this story

Dark matter, as we've recently seen, is necessary to get our models of the Universe to work. There's also extensive observational evidence for its existence, and various evidence indicates that it takes the form of heavy particles. In contrast, the evidence for dark energy comes from a single type of observation, and we have little or no idea what it might actually be. Nevertheless, the evidence for its existence has been so compelling, and so completely changed the way we view the Universe, that the Nobel Prizes in Physics this year went to members of the teams that first developed a compelling case for dark energy.

So, how do you develop evidence for something you can't understand? The answer goes back to Einstein's publication of general relativity and the first efforts to use it to analyze the structure of the Universe. At the time, the Universe was thought to be static, but relativity suggested that it could also expand or contract. To balance things out, Einstein added a cosmological constant, representing the energy of empty space, which he set to a value that would ensure the Universe remained static. A few years later, Hubble and others discovered that distant objects were moving away from the Earth at higher rates than ones nearby (as measured by greater redshifts in the light they emit). This indicated that the Universe was expanding, so Einstein removed his constant, calling it a blunder.

In the ensuing decades, the inflationary Big Bang model was developed, and it neatly explains most of the features of the modern Universe. One of its predictions, however, is that the gravitational pull of the Universe's matter (dark and otherwise) should be exerting a pull that counteracts the Universe's expansion. That pull should be subtle, but could be visible over great distances as a slight decrease in the rate of the expansion of the Universe. Distant objects will still have their light shifted toward the red end of the spectrum, but not quite as much as we'd predicted they should.

To search for this effect, The Supernova Cosmology Project was formed, led by Saul Perlmutter of Berkeley. The team used newly available digital hardware to perform surveys to identify distant Type Ia supernovae. These events, caused by the explosion of a white dwarf star, all occur when the star reaches a specific mass, and thus have similar intrinsic patterns of brightness, allowing them to act as what are termed standard candles. This brightness lets researchers determine the distance to these objects without relying on the red shift. That allowed the SCP to compare the actual redshift of these objects with the redshift calculated from their distance. Any difference between the two could show the sort of deceleration the team was expecting.

As they went about developing their techniques, one of their early papers fell across the desk of a Harvard astrophysicist named Robert Kirshner, who was asked to act as a peer reviewer. Kirschner apparently didn't think much of the work, and rejected the paper. But he thought the idea behind the work was worth pursuing, and before long, a competing collaboration, the The High-z Supernova Search, was set up, led by Brian Schmidt, an American now based at the Australian National University. Schmidt's team was behind in terms of data gathering, but they felt that they had a superior approach to analyzing the data.

Through a mix of collaboration and competition, the two teams built up a database of distant Type Ia supernovae and began to publish their results and present them at meetings. To the surprise of nearly everyone, the teams failed to turn up any hint of a deceleration in the expansion of the Universe. Instead, both teams independently found that the Universe's expansion is accelerating—we're entering a new period of inflation.

There are still some arguments over which team deserves credit for getting the answer first (see the comments on the piece linked above, which clearly indicates Kirshner is still touchy about it). But the Nobel Committee has correctly recognized that a startling result like this wouldn't have been accepted so rapidly by the physics and cosmology communities if there hadn't been independent confirmation. It took two compelling studies to build a scientific consensus that accepted a new view of the Universe, and the leaders of both teams (Perlmutter and Schmidt) have been recognized. The High-z team may have gotten a bit of extra credit for having had the first publication out, as the post-doc who performed much of its data analysis, Adam Riess, is the third honoree (he's now faculty at Johns Hopkins and works at the Space Telescope Institute).

In the intervening years, independent evidence has indicated that the measurements from supernovae have it right. Both the spatial distribution of galaxies and the cosmic microwave background indicate that there's a lot of something in the Universe (over 70 percent) that is neither dark matter nor regular matter.

Even though the scientific community has accepted what the supernova teams have found that the Universe is telling us, nobody's quite sure what the Universe means. It's possible that Einstein was right in terms of suggesting a cosmological constant. A constant repulsion could have been masked by gravitational attraction while the Universe was younger and more dense. Now that it's sufficiently diffuse, the repulsive force is able to dominate, initiating a new inflation. Unfortunately, the best prospect for a constant source of energy, quantum vacuum fluctuations, provides too much of it—in fact, it's higher by a factor of 10120.

An alternative is that the Universe has recently started experiencing a new force, a cosmological non-constant that goes by the generic term quintessence. There are some ideas for what these might be, such as quantum phase transition, but no experimental evidence for them.

The relevant scientific communities seem to have a mix of feelings about these results. There's a bit of frustration, since it's difficult to know how to start looking for something that you don't understand. But there's a confident excitement that someone will eventually figure it out, and physics and cosmology will be changed forever when they do.