Much of the book is an excellent explanation, from the beginning, of a significant part of the current state of cosmology. It does a good job of even-handedly explaining the controversy over the scientific status of inflation and the multiverse, giving Paul Steinhardt’s views equal billing with those of multiverse enthusiasts like Guth and Linde. It’s written from the point of view not of a theorist, but of an observational astronomer, and thus explains well some of the details of the current state of the technology being used. Much of the book is about the BICEP telescope, operated in the hostile environment of the South Pole.

One of the strongest aspects of the book is that it is also the memoir of a life and a profession, giving a very personal take on what it’s like to get interested in astronomy as a kid, then grow up and pursue a career in the field. Keating’s book is very much in the tradition of Watson’s The Double Helix, giving a portrayal of himself and others that doesn’t leave out the very human aspects of ambition, competitiveness and jealousy.

Unlike the Watson book, which is about a great scientific achievement, the unusual aspect of Keating’s story is that what he was involved in was not a success, but the biggest fiasco in the history of his field. On March 17th, 2014, the New York Times reported on its front page that Space Ripples Reveal Inflation’s Smoking Gun, and this same story was reported by most media outlets. This was based on results from the BICEP2 telescope unveiled at a press conference at Harvard (press release First Direct Evidence of Cosmic Inflation). At the press conference, PI John Kovac claimed that the chance the results were a fluke was only one in 10 million.

I wrote several blog postings about the story as it evolved, you can find them here, here, here and here. The BICEP2 result was often portrayed as a definitive experimental vindication of the multiverse, which was one reason I was writing about it. By the later postings, I was covering the story of the collapse of the BICEP2 claims, as it became clear that what they had measured was a signal coming from dust in the galaxy, not from primordial gravitational waves.

Keating’s insider account of what happened makes clear that the true story is that the BICEP2 telescope, because of the way it was designed (sensitive to only one part of the sky at one frequency), was never capable of distinguishing primordial gravitational waves from dust. They were in hot competition with the Planck satellite collaboration, which did have the capabilities needed to distinguish the signal they were seeing from dust, and was generally assumed to be the experiment with the best chance of seeing primordial gravitational waves. BICEP2 could have released its data, making clear that it might be primordial gravitational waves or it might be dust, that Planck would need to weigh in to decide. This would have made a splash, but probably not a front-page one, and if the gravitational wave signal was real, Planck would have shared in the glory of identifying it.

Instead of behaving responsibly, the BICEP2 collaboration found arguments to convince themselves that the dust could not be a problem, arguments which included scraping data off a slide of a preliminary Planck result presented at a conference (while, it seems, misunderstanding the significance of the data in that slide). Keating gives a very defensive explanation of how this happened, claiming that he was well aware of the danger that the signal was just dust. About Planck, he writes

We desperately tried to work with the Planck team, while being careful not to tip them off as to what we’d found… The Planck team wouldn’t cooperate.

which I guess really means “we desperately tried to rip them off, but they weren’t that dumb.” While he had these concerns, in the end he decided to agree (as did the whole collaboration) with the tactic of writing a paper claiming dust wasn’t a problem and going public with an aggressive and heavily promoted discovery claim.

The cost/reward computation they were engaged in when they decided to go public with a problematic claim involved two possibilities:

Planck data would show the dust was not a problem. If this was the case, BICEP2 would be the people who found the primordial gravitational waves, Planck the losers who measured some boring dust.

Planck data would show that the signal was dust. This would be embarrassing, but, this is America, and all publicity is good publicity, right?

As far as I can tell, the BICEP2 scientists haven’t suffered much professionally from the fiasco. When David Spergel talked here at Columbia about the subject, he noted that this hadn’t stopped the PI, John Kovac, from getting tenure at Harvard. In the book, Keating mentions some “embarrassment and guilt”, but no negative professional consequences, instead explaining how a few months later Jim Simons came to him to offer to fund a next generation observational program (the Simons Observatory, of which he is now Director) to be built in Chile. The Nobel Foundation in 2015 was contacting him to request him to nominate candidates for 2016. Keating does write that he thinks the BICEP2 story shows that scientists should be given some formal training in ethical norms, but at the same time he makes clear that violating such norms sometimes provides significant rewards, with few penalties.

A major theme of the book is Keating’s obsession with the possibility of winning a Nobel Prize as well as long discussions of what’s wrong with the way Nobel Prizes are awarded and what he feels should be done about this. On some of these issues I agree with him. In particular, the Higgs discovery story makes clear the problem with awarding prizes only to individuals, not collaborations. You end up with a prize not for the most important experimental discoveries in physics, but for the most important discoveries made by experimental groups with a small enough list of high profile leaders.

Sabine Hossenfelder’s review was quite hostile about Keating’s complaints concerning how the Nobel Prize is operated, for reasons I didn’t understand until I read the book. It’s very hard to have much sympathy for Keating’s recounting of his many Nobel-related jealousies and resentments. In particular, even if there had been no dust, he was never really in the running for a piece of the BICEP2 prize nomination since he had been pushed out of a leadership role due to his involvement with a competing experiment. He still seems bitter about this and gives the impression that this is at the root of his complaints about the Nobel. One suspects that if there had been no dust and he had been given more prominence in BICEP2, after his trip to Stockholm he’d instead have written a book describing the Nobel Prize as the most well-designed and enlightened thing in the world. Instead of owning up to mistakes and writing a post-mortem about lessons learned and what to do about them, Keating’s choice to instead write a book blaming the Nobel Prize committee is a peculiar one.

Update: Not content with using notoriety achieved through incompetent and unethical scientific behavior to launch a bizarre and incoherent campaign against the Nobel Prize, Keating is taking to right-wing media outlets to attack the atheism of his fellow scientists, see here and here. I’m afraid that on the multiverse issue he has the stronger argument: multiverse proponents are making a huge mistake using that to go to war with religion.

39 Responses to Losing the Nobel Prize

It is interesting to compare BICEP2 to OPERA, from the experimental angle.
OPERA made a highly non-trivial error, which took months of expertise to understand properly. Their release was relatively cautious (inasmutch as it is possible with such a claim), yet when the error was found the collaboration really suffered (the spokesman had to resign). BICEP2… well, the blogpost above says everything that need to be said.
Why such a difference?

With regard to BICEP2 and OPERA, part of what gets overlooked in some of this is that BICEP2 had done lovely work — they had great data at far higher sensitivity than any previous polarisation experiment, if I am recalling things correctly; the analysis was flawed but the data was good.

Conversely everything OPERA announced evaporated and even the hint of a claim to have found superluminal particles is a far larger deal than detecting a primordial B-mode 🙂 Not really making a judgement as to whether either outcome was fair, my own sense was that the OPERA announcement did attempt to have a dollar each way on the outcome; it was a bit too much “I’m not saying aliens” for my taste, but there are big differences and well as parallels between the two cases.

So, this is a fascinating book not for the reasons its author thinks it is.

The professional benefits that BICEP2 collaborators have received, from the spokespersons down to the last student, and the impossibility of having them retracted is phenomenally annoying. If Keating is complaining about how a single prize determines research (ehm… does it really?) then he’d better complain about tycoons determining research by handing out money to people from failed experiments. But who am I to judge a heavily promoted academic.

(PS: OPERA’s claim was a crazy one by current standards, and their release was not cautious, like, at all.)

I remember OPERA were much more cautious about their claim, raising serious doubts and asking for other groups to check their results. I remember wondering at the time why it had such a destructive effect on their team when the loose cable was found (note: a loose cable is an unfortunate accident, but making a decision to scrape data off a slide and go the max publicity route with press releases and doorstep videos is not an accident).

Truth is, sorry to say, I think they had a ton more scientific integrity than the BICEP team, hence the difference in their methods of presenting their results. I found this: “Some OPERA team members thought the whole episode had besmirched the collaboration’s reputation”, hence the resignations even though it was an accident, whereas I remember the BICEP team carrying on as if nothing bad had happened, indeed, treating it as an opportunity to push for funding for a bigger telescope. Plus an opportunity for Andrei Linde to change his story and say that B-modes were not important for inflation after all.

DrDave,
My impression is that some media outlets later put out stories like “questions raised about BICEP2 results”, placed much less prominently than the initial stories. PRL referees saved the BICEP2 people from themselves by getting them to remove the worst of their mistakes. If the referees had quickly let the paper through, likely the paper would have had to have been formally retracted.

lun,
I did some searching around for examples of any apology from the BICEP2 people or negative effects on any of their careers, could find nothing. A quite remarkable fact is that Kovac’s Wikipedia entry still has the following as description of his career:

“He is the principal investigator of the BICEP2 telescope, which is part of the BICEP and Keck Array series of experiments.[3][4][5] Measurements announced on 17 March 2014 from the BICEP2 telescope give support to the idea of cosmic inflation, by reporting the first evidence for a primordial B-Mode pattern in the polarization of the CMB.[6][7][8] If confirmed, this measurement provides a direct image of primordial gravitational waves and the quantization of gravity.”

This is a fascinating blog post. I am certainly no expert on astrophysics so I can’t weigh in on the minutiae. I do wonder though if it would have been possible for the BICEP2 folks to have a third party review their work and it’s extraordinary claims before submitting it formally to the peer review process. I get the sense from the blog post that they were far too deep into their scientific process to be throughly objective about drawing conclusions. Not being critical, just wondering if an aloof reviewer might have helped shepherd the conclusions a bit more. Perhaps this was not a realistic thing to do secondary to time and publishing pressure…..not to mention Nobel aspirations!!

Tony Verow,
Keating doesn’t really address at all the question of why the dust estimates BICEP2 used were so wrong. Part of the story is surely that they wanted to believe low dust estimates.
Since their experiment was inherently incapable of distinguishing dust, likely they themselves had little expertise in the dust issue. They should have been getting outside help on this, but perhaps the people with this expertise pretty much all were associated with competing experiments like Planck that could do dust measurements. BICEP2 did not want the competition to get wind of their result, out of fear that they would get scooped, so they weren’t about to consult outsiders.

My recollection of the BICEP2 thing is perhaps different from others. I remember the repercussions of the episode more clearly than the initial announcement.

I’m thinking of seeing some videos of “random knocks on the door” of people in Palo Alto (Linde and his wife) which were clearly staged (they both appeared at the door) and then the denoument (sp?) a few weeks later that the results were not what they had initially been thought to be.

Tim May,
The video you are thinking of is here:https://www.youtube.com/watch?v=ZlfIVEy_YOA
It was put on YouTube by the Stanford press office, has over 3 million views. It is still there with original text claiming discovery, has not been updated with the news this was just dust. The same is true of the linked Stanford web page with story for the public and the press (no retraction there). This page links to the Bicep/Keck collaboration web page. Nothing on their web page anywhere acknowledging that their original claims were a mistake.

Yes, science worked, but with no help from the BICEP2 people. Their attitude seems to have been (and continue to be) that there is nothing wrong with making highly publicized and hyped discovery claims that turn out to be bogus, because other people will clean up your mess if you are wrong (and, it’s all the Nobel committee’s fault for making you do this…).

I’ve always felt a little bad for Linde, though. I have my issues with various of his behaviors, but it does appear to me (unlike Tim May) that, in this instance, he was genuinely surprised, and on video, and that he actually tried a little bit to be cautious (“Let us hope it is not a trick,”) and still to some degree got laughed at in public through actions entirely of others.

It’s always best to simply present your result in the literarature, not to push it too hard, let momentum build over time as it’s importance is understood. There’s nothing worse than some scientists endlessly trying to sell their result, which they are convinced is so important. We’re not salesmen, we’re scientists. We’ve got class.

I’m with Tim May. Exhorting one and all to meet the highest ethical standards is a worthy cause that I won’t debate. But the practical reality has always been, and likely will always be, that if an incentive structure exists that rewards bad behavior, then there will be bad behavior. Science often works well when human competitiveness (which is also part of the bad incentive structure) can be harnessed as a kind of innate immunity…so long as there is an objective standard by which the results of competitors can be judged. To the extent we continue to value accurate accounting of natural phenomena over some alternative, scientific inaccuracy is something humanity can cope with.

Of course, it helps a lot when the natural phenomena in question can be observed. Lose observable reality, in the conventional sense, and you lose science, as far as I’m concerned. Here, fortunately, science really did work. It doesn’t have to be pretty.

My problem really isn’t with the ethical issues of how scientists compete with one another to do science (although, when caught very publicly doing something unethical, it would be nice if the people involved would at least say they were sorry…). The unusual part of this story was the full-court press by scientists and their institution’s press offices to promote a bad scientific result and get it on the front page of the New York Times, coupled with a seeming complete lack of concern once it became clear that what was being promoted was an epic fail. For instance, I don’t think there’s any conceivable excuse for the BICEP2 people and the Stanford press office not taking down that YouTube video, or at least editing the associated text to explain what happened.

This isn’t complicated. If you submit a paper to a high-profile journal, get it accepted, put out a snazzy press release to accompany the publication of the paper, and then it all ends up falling apart because the results evaporate, *THAT* is how science works – and such a route is still filled with large egos and dreams of glory.

If you announce the result via a press conference *BEFORE* having the paper accepted in a journal then *you have to be correct* and there are no excuses for error. There can’t be any pleading afterwards that this is the scientific process in normal operation, because it’s not.

This whole episode was an embarrassment and it should have had measurable negative effects on the careers of those involved. It didn’t.

srp,
I don’t think the problem is the press office, it’s the BICEP2 scientists. Looking through all their various current webpages about their work, it’s nearly impossible to find anything that explains their heavily publicized results were wrong. The closest thing I could find was this:https://kipac.stanford.edu/research/projects/bicepkeck-array
which tells us
“BICEP2 detected this B-mode polarization in its three year data set with a strong significance. Interpreting this as primordial gravitational waves, this places a constraint on the tensor to scalar ratio at r=0.20+0.07/-0.06. This is strong evidence for cosmic inflation, as well as the first direct image of gravitational waves, and direct empirical evidence of quantized gravity. However, the joint analysis with Planck satellite data followed and showed that the galactic dust seems to contribute much of this B-mode polarization measured by BICEP2. ”
No admission there was any problem with what they did in March 2014, and “seems to contribute much” is highly misleading.

To be clear: I pretty much agree with everything said. There’s no excuse.

Maybe my expectations have hit rock bottom, but I’m encouraged that we aren’t in a branch of the multiverse where BICEP2’s results are being touted as proof thereof, or anything else of interest beyond cosmic dust backgrounds. There was a better experiment, a joint analysis, and no one need consider the question of primordial B-mode polarization in the BICEP2 data set ever again.

The fact that “science worked” is still a remarkable thing, I guess. Given that Keating and his ilk really are out there, it’s something to be thankful for, at least a little.

I am reminded of a story I once heard about Leo Kadanoff. Apparently, on the day Ken Wilson was awarded the Nobel prize, Kadanoff showed up in the department wearing a three-piece suit. When asked why he was wearing such a thing, he laughed and replied, “Well, it’s not every day you don’t win a Nobel Prize.”

The fiasco was partly bad luck. The Planck-scraped data were preliminary results regarding dust levels, based on an area of the sky far from where BICEP2 was looking. There were no criticisms until more preliminary Planck information revealed that other areas had somewhat higher dust levels, generating warning flags to the BICEP2 extrapolation. Ultimately, the dust levels where they were looking turned out to appear at the same magnitude of the purported B-modes.

However, it is just as premature to state that BICEP2 actually observed dust as it had been to claim they observed B-modes. It is currently unknown what the BICEP2 data represents, as they only measured one frequency. BICEP3 is measuring several frequencies, and combined with the BICEP2 data, they should be able to tease out the difference between B-modes and dust.

As for ethics, that’s one big minefield. I neither agree nor disagree with the ethical criticisms of the BICEP2 collaboration. There are those that claim that all data, all preliminary deductions, and so on, ought to be shared promptly with the scientific community, instead of hoarded to maximize the original researchers’ return on investment, as is commonly done. From that point of view, the Planck collaboration was in the wrong.

That radio talk is pretty disturbing – he is hinting that scientific bias and groupthink of his colleagues in cosmology are actually rooted in their leftist politics and their godlessness. Then he explains how the Nobel committee is so non-transparent and possibly also unethical, and unwilling to listen. He goes on and on how Nobel is essential to the right functioning of science, and how he was elbowed out from the BICEP collaboration by his fame-thirsty colleagues who wanted maybe the Nobel for themselves… And I don’t understand what Torah has to do with anything in his field unless he wants to win Templeton instead.

Kadanoff’s statement seems to be missing “… when you expected to” at the end – I expect every day of my life to be a day that I do not win a Nobel Prize, and yet rarely wear a three-piece suit.

I cannot speak for other fields, but definitely feel that exaggerated regard for this particular trophy is part of the problem in high-energy physics. Following the 1979 Glashow, Salam, Weinberg award HEP theorists seem to have wasted inordinate amounts of time building models as Nobel Prize lottery tickets instead addressing the real underlying problems of quantum field theory.

william e emba,
According to Spergel and others, the problem with the BICEP2 use of the Planck preliminary data was not that it was preliminary, but that they ignored the notation “not CIB subtracted” on the slide. If they had taken that into account, they would have found that it implied their result could easily be just dust. This wasn’t bad luck, it was incompetence.

What BICEP2 was trying to do with Planck was get access to their unpublished data that they needed, without them finding out that they had interesting unpublished data. You can make the argument that all experiments should share unpublished data, but that’s exactly what BICEP2 was trying to avoid (what they ended up with, a shared analysis with Planck). Yes they were unlucky, in the sense that they didn’t get away with this.

The attempt to argue that maybe BICEP2 was right is nonsense. They claimed observation of r=.02, not sure what current limits are, but they’re below r=.01.

That is disturbing. Watching things in this field go from bad to worse, ending up with multiverse mania, I’ve always thought something like “at least it can’t get worse than that”. This makes clear there is something worse possible: multiverse pseudo-science becoming part of our culture’s sad descent into tribalism and culture wars.

“I’m afraid that on the multiverse issue he has the stronger argument: multiverse proponents are making a huge mistake using that to go to war with religion.”
Are “multiverse proponents” really doing this? From the video you linked it seems more like Keating is the one building a fake dichotomy between the multiverse and god.

Humans will be human. Nobel Prizes are a sociological phenomenon… not a scientific one. Science doesn’t need them, but they do motivate some humans to discovery, and motivate other humans to jealousy.

Humans, not science, decide funding priorities.

Was it Benjamin Graham, a stock analyst, who said… in the short turn, the stock market is a popularity contest, and in the long run, a weighing machine? In the short run publicity and politics do decide science funding and priorities, in the long run, science *should*. And, I might add, in the long run, we are all dead.

Short run decisions will always be error-prone. It is up to all of us to focus on the science in the long run, and get better results. And seems to me in the B-modes, science did win in the end. I can forgive and overlook all the wrong BICEP hype… it is part of the short-term transient of human error.

I think Keating and BICEP also warrant a fair amount of forgiveness, and it doesn’t really bother me that Kovac has tenure.

Fermi’s Nobel Prize was partially for the discovery of the new elements Hesperium and Ausonium, which were flat wrong. Ida Noddack even pointed out the flaw in Fermi’s arguments, prior to the Nobel Prize. Science lost in the short term then.

But Fermi’s techniques of neutron bombardment still were a stunning advance in science. BICEP also got the ball down the field.

Keating ain’t my favorite guy. As the old saying goes, though, sometimes it takes a pig to find a truffle. But, as truffle hunters know, the pig usually eats it, which is why dogs are used by truffle hunters. Yup, we experimental scientists are just pigs and dogs.

But in the long run, with concerted effort, a few truffles can be found, not eaten, and actually discerned and appreciated by everbody. I hope.

Well, then, when hyping their BICEP2 results in a scientific journal and in the scientific press and in science departments, the Kovac et al. should have stated that they arrived at the results not as scientists, but as humans. They should have warned the unsuspecting public.

Perusal of the “educational” channels on TV, e.g. AHC, will show the multiverse frequently being used as part of the large pagan web being spun by those channels. In brief, multiverse beliefs are used to legitimize and “scientize” Heaven, the Hereafter, alternate realities, etc. This web of contrarian belief, typified by the vast mythology of the Ancient Aliens series, is a continuation of the New Age writings of the 1970’s. Historians will note that all of this is a recurrence of the Spiritualism of the early Twentieth Century. But like a previous commentor, I see little warrant for blaming the multiverse proponents for such exploitation.

Stephan said: “Well, then, when hyping their BICEP2 results in a scientific journal and in the scientific press and in science departments, the Kovac et al. should have stated that they arrived at the results not as scientists, but as humans. They should have warned the unsuspecting public.”

I’m not aware of a single scientific result that was arrived at by anybody other than a human. Therefore, I don’t think there was any need for the BICEP2 folks to make a distinction. We all know they are merely error-prone humans. We all know science is done exclusively by error-prone humans. I tend toward forgiveness for the errors. Stephan, if you don’t want to forgive, that is fine, your choice.

S,
Gross, like many string theorists, has a long history of arguing that “spacetime is doomed”, that string theory will lead to some new fundamental theory in which spacetime is not fundamental, but an emergent concept. I think he’d readily admit though that there never has been a convincing specific implementation of this idea. These days people are trying to get spacetime as emergent using entanglement/holography ideas, I’d be curious to hear his take on those.

Most criticism of the Nobel prize is just based on simple misunderstandings. No, the Nobel Foundation would never ever send letters to anyone asking them to nominate. The Nobel Committees (one for each prize) do that, and they send out thousands of such letters, so getting one doesn’t imply anything… Finally, the Nobel prizes are awarded according to a set of rules and criteria. The most important are the ones specified in the will of Alfred Nobel. These obviously cannot be changed! The will forbids awarding groups or institutions, the peace prize being an exception. In fact, according to the wording of the will, it is not the people that are being awarded, it is the discoveries. When the Nobel committees decide on the prizes, they first decide which discovery shall be awarded, and only then do they decide on the individual(s).
Besides the will itself, there is an agreement between the executor of the will and the Nobel family, which clarifies some things that the will does not make any recommendations on. For instance, it was agreed that a prize can be split between two or three individuals, and it sets out rules for who are eligible to nominate.
The purpose of the Nobel prize is to honor the last will and testament of Alfred Nobel. It makes no sense for random people to have opinions about how he wrote it.

I remember the URSI meeting in Boulder in I think January 2012, when I counted at least 9 different experiments all trying to measure B-mode polarization in some form, through different means. I thought (and still think) this was an absurd allocation of resources, but then I don’t get to make such calls.

The two big considerations were what l (basically angular frequencies) you’d be sensitive to (Planck and BICEP were complimentary in this regard), with what colors, and how well you could eliminate the foreground dust. One experiment’s goal, and it may have been BICEP/Keck Array, was to just measure the B modes first. If they could, then they could figure it out later if the signal was from dust or not. If you couldn’t, you put an upper limit on what was there to be detected. I don’t think anyone in that room thought it really would have even been detected within two years. I left my somewhat related field permanently shortly thereafter

I was very surprised by the early 2014 announcement. Having known and worked nearby a lot of the members, though not Keating, I thought there was no way they would make such a claim without thorough rejection of the dust hypothesis. Everyone I knew thought the B-mode signal from dust would dominate any cosmic signal. So I thought they must have very strong evidence, but, having left the field already, I really couldn’t investigate. I must say, the unraveling did not surprise my intuitions, but it did surprise me that the people who made the claims had done so with such disregard for prudence and caution.

I think a lot of this had to do with 9 separate, or largely separate, groups all doing the same thing. How in the world is having so much funding going into these areas a good thing? Everyone just picked up on the hot topic of the day and raced into it. Any idea how to fix that?