In 1911 and 1912, Austrian physicist Victor Franz Hess made a series of ascents in a hydrogen balloon in a quest to find the source of ionizing radiation that registered on an electroscope. The prevailing theory was that the radiation came from the rocks of the Earth.

During the last of his seven flights, Hess ascended to more than 5,300 meters – almost 17,400 feet – to find that the rate of ionization was three times of that at sea level. Hess concluded that the upper atmosphere is ionized by radiation from space, not the ground, and proved that this radiation is not solar after conducting experiments at night and during eclipses. Hess had in fact discovered cosmic rays, and was awarded the Nobel Prize in Physics in 1936.

Since being first postulated by theoretical physicist Wolfgang Pauli in 1930, cosmologists have been hunting for neutrinos – subatomic particles that lack an electric charge, and once described by Nobel Prize winning physicist Frederick Reines as “the most tiny quantity of reality ever imagined by a human being.”

Now, an international team of scientists has found the first evidence of a source of high-energy cosmic neutrinos, subatomic particles that can emerge from their sources and, like cosmological ghosts, pass through the universe unscathed, traveling for billions of light years from the most extreme environments in the universe to Earth.

“This result is the first of its kind,” declared National Science Foundation Director France Córdova in announcing the finding at a July 12 press conference. “We have never before used multi-messenger astrophysics to pinpoint the origin of high-energy cosmic rays.”

The detection was made last September 22 by an international team at the NSF-funded IceCube Neutrino Observatory, an array of 5,160 optical sensors deep within a cubic kilometer of ice at the South Pole. The findings were confirmed by telescopes around the globe and in Earth’s orbit, according to the announcement.

The neutrino that alerted telescopes around the world had an energy of approximately 300 TeV (teraelectron volt), which is about 50 times larger than the highest energy terrestrial accelerator ever built. The scientific importance of this observation lies in the fact that the source for such a high energy cosmic ray could be convincingly identified as a known blazar, roughly four billion light years from Earth. A blazar is a giant elliptical galaxy with a massive, rapidly spinning black hole at its core.

Observing high energy neutrinos that originate from such a blazar simultaneously implies a dual discovery. It identifies blazars as at least one of the sources of high energy neutrinos observed on Earth, and it identifies blazars as proton accelerators because neutrinos come from pion decay, requiring proton acceleration to be produced.

Comet, Open Science Grid

Assisting in the latest discovery was the petascale Comet supercomputer, based at the San Diego Supercomputer Center at UC San Diego, funded by the NSF in late 2013. Comet is capable of an overall peak performance of two petaflops – or two quadrillion calculations per second.

“The evidence for the observation of the first known source of high-energy neutrinos and cosmic rays is compelling,” said Francis Halzen, a University of Wisconsin–Madison professor of physics and the lead scientist for the IceCube Neutrino Observatory. “This has been one of the oldest open questions in astronomy.”

In an earlier interview with SDSC, Halzen explained the importance of Comet for isolating the signature pattern of neutrinos: “The IceCube neutrino detector transforms natural Antarctic ice at the South Pole into a particle detector. Progress in understanding the precise optical properties of the ice leads to increasing complexity in simulating the propagation of photons in the instrument and to a better overall performance of the detector.”

The photon propagation in the ice is very well-suited to run using graphics processing units (GPUs) hardware such as those on Comet, Halzen said. “Pursuing efficient access to a large amount of GPU computing power is therefore of great importance to ensure that future IceCube analysis reaches the maximum precision and that the full scientific potential of the instrument is exploited.”

Following the September 22 detection, the IceCube team quickly scoured the detector’s archival data and discovered a flare of over a dozen astrophysical neutrinos detected in late 2014 and early 2015, according to the NSF announcement.

“These intriguing results also represent the remarkable culmination of thousands of human years of intensive activities by the IceCube Collaboration to bring the dream of neutrino astronomy to reality,” said Darren Grant, a professor of physics at the University of Alberta and the spokesperson for the IceCube Collaboration, an international team with more than 300 scientists in 12 countries.

SDSC recently doubled the number of graphic processing units (GPUs) on Comet in direct response to growing demand for GPU computing among a wide range of research domains. The expansion makes Comet the largest provider of GPU resources available to the NSF’s eXtreme Science and Engineering Discovery Environment (XSEDE) program, a national partnership of institutions that provides academic researchers with the most advanced collection of digital resources and services in the world.

In terms of allocations, Comet has been one of the most used high-performance computing (HPC) systems by IceCube researchers. Since the start of 2016, more than 1.6 million core hours of CPU (central processing unit) time has been used by IceCube researchers, and another 330,000 hours of GPU time. The GPUs are significantly faster for this type analysis, and 330,000 hours is roughly equivalent to 4.6 million CPU hours. More information about how supercomputers and SDSC in particular are playing a critical role in data-enabled since can be found in this feature article.

Moreover, allocations for supercomputing time on several systems including Comet via the Open Science Grid, a multi-disciplinary research partnership specializing in high-throughput computational services funded by the U.S. Department of Energy and NSF, has totaled almost 12.5 million core hours since the start of 2016.

Through a partnership with XSEDE, OSG scientists have access to resources such as Comet to further their research. The integration of Comet into the OSG provisioning system was led by a team including Frank Würthwein, SDSC’s lead for distributed high-throughput computing and an expert in experimental particle physics and advanced computation. Würthwein also serves as OSG’s Executive Director.