Research spurs innovations in computing technology that drive advances to supercomputers

March 29, 2012

This diagram illustrates the wide range of distance scales that must be understood before the kaon-decay calculation can be performed. The lowest layer is a picture showing the tracks of the decay particles as they move through the liquid hydrogen of a "bubble chamber" — a kind of particle detector used in the 1950s and 60s. The next layer is a diagrammatic interpretation of what's happening in the bubble-chamber picture — how the kaon (K) is produced and "breaks apart" to form two other particles: the positive pion (π+) and negative pion (π -). This process happens on the familiar scale of a fraction of a meter. The next scale of a few femtometers is shown on the third layer, where the lattice of points and paths represents the supercomputer calculation, which takes into account the binding of quarks and antiquarks as they form the particles being studied. Finally the top layer shows what is known as a Feynman diagram of the shortest scale — 1/1000 of a femtometer — the scale at which a quark undergoes a sort of metamorphosis from one flavor into another.

UPTON, NY — An international collaboration of scientists has reported a landmark calculation of the decay process of a kaon into two pions, using breakthrough techniques on some of the world's fastest supercomputers. This is the same subatomic particle decay explored in a 1964 Nobel Prize-winning experiment performed at the U.S. Department of Energy's Brookhaven National Laboratory (BNL), which revealed the first experimental evidence of charge-parity (CP) violation — a lack of symmetry between particles and their corresponding antiparticles that may hold the answer to the question "Why are we made of matter and not antimatter?"

The new research — reported online in Physical Review Letters March 30, 2012 — helps nail down the exact process of kaon decay, and is also inspiring the development of a new generation of supercomputers that will allow the next step in this research.

"The present calculation is a major step forward in a new kind of stringent checking of the Standard Model of particle physics — the theory that describes the fundamental particles of matter and their interactions — and how it relates to the problem of matter/antimatter asymmetry, one of the most profound questions in science today," said Taku Izubuchi of the RIKEN BNL Research Center and BNL, one of the members of the research team publishing the new findings. "When the universe began, did it start with more particles than antiparticles, or did it begin in a symmetrical way, with equal numbers of particles and antiparticles that, through CP violation or a similar mechanism, ended up with more matter than antimatter?"

Either way, the universe today is composed almost exclusively of matter with virtually no antimatter to be found.

Scientists seeking to understand this asymmetry frequently look for subtle violations in predictions of processes described by the Standard Model. One property of these processes, CP symmetry, can be explored by comparing two particle decays — the decay of a particle observed directly and the decay of its anti-particle, viewed in mirror reflection. "C" refers to the exchange of a particle and its antiparticle (which is exactly the same but with opposite charge). "P" specifies the mirror reflection of this decay. But as the Nobel Prize-winning experiments showed, the two decays are not always symmetrical: In some cases you end up with extra particles (matter) and CP symmetry is "violated."

Exploring the precise details of the kaon decay process could help elucidate how and why this happens.

Supercomputing the decay process

The new calculation of one aspect of this decay, which required creating unique new computer techniques to use on some of the world's fastest supercomputers, was carried out by physicists from Brookhaven National Laboratory, Columbia University, the University of Connecticut, the University of Edinburgh, the Max-Planck-Institut für Physik, the RIKEN BNL Research Center (RBRC), the University of Southampton, and Washington University. The calculation builds upon extensive theoretical studies done since the first 1964 experiment and much more recent experiments done at CERN, the European particle physics laboratory, and at Fermi National Accelerator Laboratory.

The unprecedented accuracy of the measured experimental values — which incorporate distances as minute as one thousandth of a femtometer (one femtometer is 1/1,000,000,000,000,000th of a meter, the size of the nucleus of a hydrogen atom) — allowed the collaboration to follow the process in extreme detail: the decay of individual quarks (the subatomic components of many Standard Model particles) and the flitting in and out of existence of other subatomic particles. Viewing the picture from farther away — a few tenths of a femtometer — this basic process is obscured by a sea of quark-antiquark pairs and a cloud of the gluons that hold them together. At this distance, the gluons begin to bind the quarks into the observed particles. The last part of the problem is to show the behavior of the quarks as they orbit each other, moving at nearly the speed of light through a swarm formed from gluons and further pairs of quarks and antiquarks, and at last forming the pions of the decay under study.

To "translate" the mathematics needed to describe these interactions into a computational problem required the creation of powerful numerical methods and advances in technology that made possible the present generation of massively parallel supercomputers with peak computational speeds of hundreds of teraflops. (A teraflop computer can perform one million million operations per second).

The actual kaon decay described by the calculation spans distance scales of nearly 18 orders of magnitude, from the shortest distances of one thousandth of a femtometer — far below the size of an atom, within which one type of quark decays into another — to the everyday scale of meters over which the decay is observed in the lab. This range is similar to a comparison of the size of a single bacterium and the size of our entire solar system.

The collaboration carried out the computation using the methods of lattice quantum chromodyamics (QCD — the theory that describes fundamental quark-gluon interactions), in which the decay is "imagined" as taking place within a lattice or grid of space-time points that can be entered into a computer. Then, the quantum fluctuations of the decay are calculated by a statistical method called the "Monte Carlo" method, which provides the most likely of the fluctuations as a result. The calculation required 54 million processor hours on the IBM BlueGene/P supercomputer installed at the Argonne Leadership Class Facility (ALCF) at Argonne National Laboratory. Earlier calculations were also done on Brookhaven's QCDOC (for QCD on a chip) supercomputer, a prototype for IBM's BlueGene series.

This calculation, when compared with predictions from the Standard Model, allows the scientists to determine another remaining unknown quantity important to understanding kaon decay and its relation to CP violation. A direct calculation of this remaining unknown quantity and a higher precision recalculation of the present result will be the focus of future research, requiring even more computing power.

"Fortunately," says co-author Peter Boyle of the University of Edinburgh, "the next generation of IBM supercomputers is being installed over the next few months in many research centers around the world, including the ALCF, the University of Edinburgh, the KEK laboratory in Japan, Brookhaven Lab, and the RBRC."

These new IBM BlueGene/Q machines are expected to have 10 to 20 times the performance of the current machines, Boyle explained. "With this dramatic boost in computing power we can get a more accurate version of the present calculation, and other important details will come within reach," he said. "This is a nice synergy between science and the computer — the science pushing computer developments and the advanced computers pushing science forward, to the benefit of the science community and also the commercial world."

The calculations were performed under the U.S. Department of Energy's (DOE) Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program on the Intrepid BlueGene/P supercomputer in ALCF at Argonne National Laboratory and on the Ds Cluster at Fermi National Laboratory, computer resources of the USQCD Collaboration. Part of the analysis was performed on the Iridis Cluster at the University of Southampton and the DiRAC facility in the UK. The research was supported by DOE's Office of Science, the U.K's Science and Technology Facilities Council, the University of Southampton, and the RIKEN Laboratory in Japan.

DOE's Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.