How to Be Sure You've Found a Higgs Boson

By

Carl Bialik

July 6, 2012 6:52 p.m. ET

The physicists who announced the likely discovery of the long-sought Higgs boson particle this week were operating according to an extremely high standard of certainty. As was widely reported, in order to achieve discovery status their experiment had to clear a threshold of "five sigmas" of statistical significance.

What precisely the five-sigma mark means, however, wasn't always clearly explained in the coverage of a ground-breaking development that could explain how particles have mass and, by extension, why planets and all other objects exist at all.

ENLARGE

Part of the Large Hadron Collider in Geneva that scientists used to discover the Higgs boson particle.
Agence France-Presse/Getty Images

The Numbers Guy blog

Previously in the Numbers Guy

That is partly because the five-sigma concept is somewhat counterintuitive. It has to do with a one-in-3.5-million probability. That is not the probability that the Higgs boson doesn't exist. It is, rather, the inverse: If the particle doesn't exist, one in 3.5 million is the chance an experiment just like the one announced this week would nevertheless come up with a result appearing to confirm it does exist.

In other words, one in 3.5 million is the likelihood of finding a false positive—a fluke produced by random statistical fluctuation—that seems as definitive as the findings released by two teams of researchers at the CERN laboratory in Geneva.

This is a very high burden of proof as far as science goes. For many medical experiments, researchers need merely to clear two sigmas. Sigmas don't scale in a linear way: A two-sigma result can have as much as a 5% chance of occurring as a false positive. Three sigmas, needed to cite evidence—but not discovery—of a new particle in physics, correspond to a one-in-741 chance.

More on Higgs Boson

Scientists at Europe's CERN research centre believe they have found one of the basic building blocks of the universe -- a subatomic particle called the Higgs boson. The Director General of CERN told an audience of scientists that the discovery is a milestone in mankind's understanding of nature and has opened up exciting prospects for further revelations in the field.

Physicists said they had discovered a new particle that is consistent with the Higgs boson, a long-sought particle crucial to scientists' current understanding of how the universe is built. WSJ's Gautam Naik explains its significance. Photo: Getty Images

Other fields have different benchmarks because less is at stake if a result proves faulty, or, as with pharmaceutical studies, there is more upside to moving ahead quickly with a promising result, statisticians say. "Drugs can be withdrawn, psychological experiments can be refuted, but nobody wants to see the laws of physics proved wrong," says David Spiegelhalter, a statistician at the University of Cambridge.

Sigma is the Greek letter used as a symbol for standard deviation—a measure of how far a finding departs from the expected one. The more sigmas attached to a result, the more likely it is significant and not due to chance. Say a series of experiments on a randomly chosen coin involves flipping it 1,000 times and then flipping it 1,000 times again and again. The average number of heads should be 500, but some experiments will yield more and some fewer. A five-sigma finding would be 590 heads.

The CERN finding, then, is the equivalent of getting a lot more heads than expected—which is unlikely to occur by fluke rather than for some systematic reason. The best explanation the CERN physicists have for this excess signal is the existence of the long-hunted Higgs boson.

They set such a high bar to rule out two other possible explanations: Either their equivalent of the coin is flawed in a way that tacks on extra positive signals; or they've run the study enough times and looked for anomalies in so many places in their data—the equivalent of running the coin experiment over and over with different coins—that they've stumbled upon a seemingly unlikely result just by looking too hard for it. Physicists call this the Look Elsewhere Effect, or LEE, and try to account for it. One CERN group said its finding's significance falls to between 4.1 and 4.3 sigmas after accounting for LEE.

Another factor can pull in the opposite direction from LEE: when another experiment finds the same thing. That bolsters a finding's significance. The Higgs boson was found at the five-sigma level by two CERN experiments.

The five-sigma requirement also helps guard against the equivalent of a faulty coin—some kind of measurement error. That appears to be the explanation for a finding last year that certain particles called neutrinos were traveling faster than the speed of light. It now appears to be the result of a flawed cable.

Particle physicists cite many examples of results that cleared three but not five sigmas and weren't replicated by follow-up studies, which helped give rise to the five-sigma rule. University of Pisa physicist Giovanni Punzi, who has collaborated on a parallel hunt for the Higgs boson at the Fermi National Accelerator Laboratory in Chicago, says physicists have debated whether the rule is "exaggeratedly high." It was devised decades ago to provide a margin of safety in case calculations of statistics for an experiment, done without today's computing power, were flawed.

But counterarguments prevailed: Modern computing power makes it easier to find false positives; and with so many particles discovered, new ones better clear a high bar. Maybe the rule is "not a bad idea, after all," Dr. Punzi says.

This copy is for your personal, non-commercial use only. Distribution and use of this material are governed by our Subscriber Agreement and by copyright law. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.