Main menu

Post navigation

Should you hold your breath for B-modes?

This morning (14th March 2014) I woke to a social media world abuzz with the exciting possibility that primordial B-modes have been detected; a result that would have a huge impact on cosmology. Let’s try to answer the question: what is the probability that such a detection has been made? As per Bayes’ theorem, the answer splits into two factors: the prior and the likelihood. The prior probabilities are relatively straight forward, but the likelihood is hard, since of course we don’t have access to the CMB data or the results! However, rather deviously, we do have access to the humans who have access to the data, and they form a biased tracer of the underlying information which we can tap into, since we understand some aspects of human behaviour fairly well.

.

Lets look at the various factors:

(1) Using humans-who-know as biased tracers of the information we don’t know

.

This worked very well for Planck, correctly predicting that N_eff would be closer to three than four, and that there would be no significant exciting new discoveries. Let’s apply this technique here. Now if the team in question had found a clean, statistically significant detection (e.g. > 4-sigma) of primordial B-modes (or any other big new result), they would probably have had to delay publication to do a lot more checks of the result to make sure they weren’t missing any systematics (in order to protect their reputations against mistakes – remember the OPERA faster-than-light neutrinos?).

.

This would slow down the whole publication/announcement process, frustrating the excited team. This extra checking is unlikely to have taken much less than a month and might take several months. Imagine walking around as a team member for several months with a >4 sigma primordial B-mode signal sitting on the tip of your tongue, waiting for final confirmation, and not being able to tell anyone!

.

Given this, we now have to ask, what is the probability that those excited team members did not tell anyone about the big discovery during that month or more, but then did let the cat out the bag in the last few days? Not very big I would argue. Apart from the fact that humans are terrible at keeping secrets, most communication is non-verbal: you can often see when a person is keeping a secret even if they don’t say anything. The team’s work colleagues would have become suspicious… Now perhaps the team is very small and are trained in the 007 techniques of keeping secrets (unlikely, they are physicists after all!), but even if that were the case, why would they have failed at the last hurdle, just days before the formal announcement?

I think a much more likely possibility is that the team have found a hint of something interesting, but not more. A hint doesn’t need to be investigated so deeply, so it would not have slowed publication much. A hint would also mean the team were not so excited about the result, so they would be able to keep their “secret” more easily.

[Update 15/3/14: one way out of this conclusion would be if the analysis was done by a very small team (two or three people perhaps) who were able to keep the secret even from the wider team, while they were doing the cross-checks. The results may have only been disseminated to the team within the last week or two, leading to the sudden spread of rumours this week.]

Now lets look at the priors. These split into two:

(2) Experimental Priors. The probability that a ground-based experiment could make a clean, statistically-significant detection of primordial B-modes given the other ground-based experimental results so far, is arguably rather small [Update 15/3/14: discussions around the experimental capabilities of BICEP2 and KECK/SPUD suggest that it is significantly more powerful than my initial prior estimates, so like any good Bayesian we should update our priors as new info comes in! You can see presentations about the experiments here and here which show how good the projections are. Whether they have reached these levels at this point is speculation.]

(3) Physics Priors: what is the prior theoretical probability that primordial B-modes are large enough to be detected by this experiment? This is tricky since there are so many inflationary models and I won’t comment further other than to say there is no particular theoretical reason to expect the signal to be at the level detectable by this experiment (an argument that has been made against spending money on chasing inflationary B-modes).

When we combine our human-based estimate of the likelihood with these priors, we are unfortunately driven towards the boring end of the spectrum. For the sake of concreteness let me bet that any interesting, robust and unexpected results are below 3-sigma, and more likely around the 2-sigma level; i.e. not statistically significant (yet). Conversely, if there is a claim of a > 3-sigma detection, I suspect that it will be found to be an undiscovered systematic or else it is one that is not robust to changes in the model assumed or priors on parameters. Finally one can be cynical and conjecture that the rumour has been carefully injected into the bloodstream of the cosmology community as a way to create excitement. Nevertheless, even in this case it is hard to imagine them calling a press conference if they had found nothing new, so we can probably put a lower limit on the “hint” of perhaps 1.5-1.8 sigma.

[Update 15/3/14: several sources now put Alan Guth and Andrei Linde at the press conference on Monday, and Andrei is scheduled to give a talk at the joint Tufts/MIT cosmology colloquium on Tuesday. This ups the ante in terms of the above lower limit. How many sigmas do we think would be needed for Alan and Andrei to be invited to, and agree to attend, the press conference? I am not sure, but presumably at least 2-sigma. ]

Of course, even a 2-sigma hint of something exciting would be wonderful (though as Roberto reminded me, a significant fraction of 2-sigma results turn out to be wrong) and it could be that everything I have written is wrong: I certainly hope this is the case…the Universe has proven to be rather boring since 2000 and we desperately need some new excitement. All will be revealed come Monday, so stay tuned!

.

* Update: 15/3/14 – Since writing my post yesterday morning a lot more details have emerged about the specifics of the rumour. In particular, the rumour suggests that a value of r=0.2 has been “detected”, which would enable them to perhaps get a 4-5 sigma detection of primordial gravitational waves. However, this would be in tension with existing results from the WMAP and Planck constraints (see fig below) which would be fascinating, but would have put the team in a serious quandary about whether to publish their results.

.

The fact that Andrei Linda is giving a seminar the day after the press conference suggests that he is particularly happy about the results and wants to emphasise the wider implications. Since he is primarily associated with chaotic inflation, that was looking rather bad after Planck, we may speculate that the new results will be more consistent with chaotic inflation and a larger value of r, and perhaps as large as the rumoured value of 0.2? Time will tell!

One case for pushing results from BICEP prematurely relative to ordinary protocols is that everyone knows that we are just a matter of weeks away from the Planck polarization data set release that implicates a determination of the same constant. If they announce now, they are the first to do so and can scoop Planck. If they wait several months, their work will be a footnote to the Planck announcement.

This consideration affects not only timing but statistical significance. Better to announce a 2.5 sigma result before Planck confirms it, than to wait to announce a 4 sigma result that confirms a similar or weaker hint from Planck that is announced first.

I don’t, however, think that they would rush to the presses with a “major announcement” with a mere result excluding any primordial B-wave signal up to the r=0.06 detection sensitivity or a bit more. The rumor is clearly that they have seen something, and I agree that nobody is going to announce with such fanfare a result of less than two sigma of a positive detection of primordial B-waves.

Another possibility is that the result has large error bars. A measurement of r=0.20 with a pretty big error bar of say +/- 0.08 would be a 2.5 sigma discovery of a non-zero value of r (and hence the rumor/PR focus on merely confirming the existence of gravitational waves rather than on saying much about particular inflation models), but would allow for a true value of r=0.04 to r=.36 at a two sigma level, which would at this point be not inconsistent with some of the current parameter space for r (and would suggest a true value at the low end).

If the result and error bar were r=0.20 +/- 0.7 then you get an almost 3 sigma discovery of a non-zero value for r and two sigma CI of 0.06-0.34, with prior results against strongly favoring a true value at the low end of the CI. This is still enough to strongly disfavor the simplest Higgs inflation models, but keeps most inflation models alive and well.