Back in September 2010, astronomers announced the discovery of a remarkable and exciting planet: it was three times our mass (high, but far closer to Earth conditions than the super-Jupiters usually found) and orbiting in the "Goldilocks zone" of its star… which meant that it could possibly have liquid water on its surface! This achingly earth-like planet made a major buzz, and in fact I used its characteristics to estimate that there could be billions of Earthlike planets in our galaxy.

But there’s just one small, really eensy-teensy problem: the planet may not exist. But it also might. Maybe.

We’re still early in the game here, and there’s a lot going on… but it’s worth peeking a bit deeper. There’s science here, and math, and even some interesting media jiggery-pokery.

Finding planets

We know of more than 500 planets orbiting other stars, and astronomers have a diverse set of tools to find them. The first were discovered by what’s called reflexive motion (a nice animation of this is on the Astrobio.net site); as a planet orbits a star, the planet’s gravity tugs on the star, causing a tiny Doppler shift in the starlight. This is a very small and difficult thing to measure, but techniques improved vastly in the 1990s, and most planets have been discovered this way. The success of this technique has been confirmed by other methods, too, including planetary transits, when the orbiting planet passes in front of the star from our viewpoint, and blocks a little bit of its light. Several planets detected using reflexive motion were confirmed by subsequent transits. We know the method works.

But like any technique, things get fuzzy when you push it. Gliese 581 is a red dwarf star a mere 20 light years way; it’s one of the closest stars in the sky to us. Two different teams of astronomers, one Swiss and one American, have observed the star for a long time, and they both confirm the existence of four planets around the star (more on that in a sec). But one of the teams (Steven Vogt and Paul Butler) claimed they found two more planets: Gliese 581 f and g, with the latter being the planet in question.

Odd planet out

Almost immediately, the planet was called into doubt; the Swiss team re-examined their data and could not be absolutely certain that Gliese 581 g was there, but still gave it a thumbs-up at the 90+% level. That’s not too bad.

Interestingly, not too long after the announcement I was at a meeting with several astronomers, and one noted that Vogt’s team made a big assumption: all the planet orbits were circular. If in fact one of the planets had an elliptical orbit it could set up a false-positive, making it look like another planet was there when it wasn’t. According to Vogt this turns out not to be the case; I contacted him and he let me know that orbital ellipticity was one of the characteristics they modeled as a variable. In other words, their computer model made no assumptions about orbit shape, but in fact the best fits in the end were circular orbits.

Still and all, there have been some questions about the planet’s existence, and I’ve been holding back from posting until something happened. Well, something did: Philip Gregory, an emeritus astronomer with the University of British Columbia, has analyzed both data sets using sophisticated statistical techniques, and he concluded that Gliese 581 g almost certainly wasn’t real. In fact, he says the odds of it being a false alarm are 99.9978%!

So which is it? Is it 90+% certain to be real, or 99.9978% certain it isn’t?Let me be up front with you: I don’t know. Gregory analyzed the data using Bayesian analysis, a method of looking at the statistical certainty of a set of observations. This is fiendishly complex in practice and to be honest is not something I’m familiar with. However, in his paper, Gregory himself claims that Vogt and Butler underestimated the amount of noise in their data. Vogt disputes this, saying that Gregory adds noise to their data rather arbitrarily. I’ll admit that it seemed odd to me that Gregory would add noise the way he did, but again I’m no expert.

Vogt also notes that how you run the computer model will change whether or not you find the planet. This part interests me, because I’ve run into similar situations myself. If you tell your computer that one of the planets (in this case, Gliese 581 d) has a highly elliptical orbit, then Gliese 581 g disappears: when you calculate the statistics, it’s far more probable that the planet does not exist. But if you keep Gleise 581 d’s orbit circular, Gliese 581 g can be seen in the data. These two different assumptions lead to two different solutions, where one has Gliese 581 g in it and the other doesn’t.

Which one is right? Vogt claims 581 g exists. I won’t go into details (the math gets a bit hairy) but basically he claims that statistically speaking, his solution fits the data batter then Gregory’s.

He said/He said

Well, that’s science! Two people disagree, and they make their cases. Vogt’s disagreements with Gregory’s methods are reasonable, in that he can make his case scientifically and mathematically. He may not be correct, but that’s a matter to be hammered out using science and peer review. Given that the claims are pretty specific (methods used, input parameters, statistical measurements), I think this will work itself out pretty rapidly.

However, the media got involved, and then things got a bit sticky.

I was tipped off to this matter with a link to the (Australian) ABC site which wrote about this disagreement. The following passage, I’ll admit, made me cringe a little. Note that the HIRES data are the observations by Vogt and Butler, while HARPS is from the other, Swiss, team:

Dr Steve Vogt says he and his colleagues "stand solidly" by their original findings.

"I have studied [the paper] in detail and do not agree with his conclusions," he says.

Vogt is concerned that Gregory has unfairly manipulated the HIRES data.

"By doing so, he finds a solution that is more consistent with the HARPS data only," he says.

OK, yikes. The word "manipulated" is pretty loaded. It’s easily interpreted as meaning the data are somehow being changed unfairly, and on purpose.

The revelation Gregory put forward is being dismissed by Vogt, who was quoted by the Australian Broadcasting Corporation as saying Gregory “manipulated” the numbers.

Egads. That made me cringe a lot. Note this is a second-generation quote; the Star was using something written in the ABC article. The Star continued with this:

"Vogt is not familiar with the Bayesian techniques so he might assume that I am manipulating the data. I attribute that to a lack of awareness on his part," said the soft-spoken Gregory.

Oh my. Well, to me the use of the word "manipulate" would be pretty accusatory in this context coming from a scientist when discussing the work of another, and this is why I initially contacted Vogt. He sent to me the email he sent to the ABC, and the word "manipulate" is nowhere in it. To a layman his email would be strongly worded, but as a scientist I see him attacking Gregory’s work, not the man himself. What he said wouldn’t draw any surprise at all were it said at a scientific conference, for example.

But the Star article actually got a response from Gregory about the "manipulation". That line I quoted above is a bit loaded, in my opinion, right down to the adjective "soft-spoken" used to describe Gregory. It’s almost as if the media were playing up the contention between the two men, trying to frame the story as being personal (with one scientist the aggressor, and the other the defender) as opposed to just a scientific difference of opinion.

Again, I strongly suspect that if Vogt and Gregory got together (or when Gregory’s paper goes through the review process; it’s been submitted but not peer-reviewed yet) this would all get figured out pretty quickly.

So, does Gliese 581 g exist? I can only form an opinion right now based on what I’ve seen, and I don’t like to speculate over much. However, Vogt has good rebuttals to the opposing claims, and the Swiss team of astronomers does seem to back him on the existence of the planet.

What we really need are more and more sensitive observations. That’s going to be the rule and not the exception as we move forward in looking for earth-like planets. They’re small, and move slowly, and make themselves very difficult to detect with our current hardware. But progress moves on, and whether Gleise 581 g exists or not, finding another Earth orbiting another star is only a matter of time. Count on it.

1. How much eccentricity in the orbit of 581f is required in order for 581g to disappear?

2. If there were intelligent observers on a planet orbiting around Gleise with identical equipment to that being used here, would they be able to detect the planets in our solar system? It would appear that all of the planets in our solar system are too far from the sun to be detected. In addition, the Sun is more massive then Gleise so that the recoil effect on the former would be smaller.

People who don’t understand Bayesian statistics (aka, the only correct form of statistics), should not be using frequentist models at all. If it is true that Voigt doesn’t understand bayesian “techniques”, then all his claims should be viewed in an extremely skeptic eye. Most frequentist “models” usually arrive at a very optimistic result, whereas such optimism has no real statistical basis (false positives).

My bets are this planet does not exist and Voigt has just fooled himself up.

At this point I’m pretty sure that the media feel that it is simply not possible to print news about a science disagreement without painting it as a personal one. It would be like a car crash in a Hollywood movie not exploding afterwards: it simply isn’t done.

There have been several papers regarding the existence of Gliese 581g that have come out since then, all re-examining the same data that has been published up until this point. (Note that the HARPS team have not yet published their extended dataset).

Of these, the one by Anglada-Escudé suggests that the data is compatible with the existence of all six planets. Score one for planets f and g.

A paper on the use of reduced chi squared by Andrae et al. points out that you cannot use reduced chi squared to do model comparison if your model is nonlinear, which the orbital models are (and Vogt et al. did use the reduced chi square statistic to settle on the six planets model). Using a statistic that is well-defined for nonlinear models, they come up with the 4 planet model being the one that is most well-supported, using the same data that Vogt’s team did.

Gregory’s analysis suggests the presence of either 4 or 5 planets, with the extra HIRES data weakening the case for three of them.

Personally I’ve taken a look using the Systemic console and reproduce Gregory’s analysis that the HIRES data alone can only find two of the planets (b and c). Using HARPS alone I can only get 4 convincing planets (the residuals to this model do not show anything significant around any of the claimed periods for f or g) and using the combination of datasets I get six. I get the same results whether I use eccentric or circular orbits. Because the two additional planets (f and g) only appear in the combination of two datasets that do not seem to contain them on their own, I currently strongly suspect they are artifacts of the merging.

I of course hope 581g exists, but then again we can’t draw any conclusions just yet. As a side note, I also am very hopeful as to the existence of Gaia Centauri – my name for any habitable zone earth-mass planets found in our nearest system – which is likely, but then again unconfirmed. But put aside bias for now, bring on the data.

Misora: A tidal lock doesn’t preclude habitability. Google “Aurelia and Blue Moon”. If atmospheric circulation is going well, the difference between temperatures on the day and night sides will be lessened. Assuming it’s there, with 581g a thick atmosphere to aid in circulation is easier to come by, given the planet’s higher mass (assuming it would begin to acquire one in the first place, and also assuming that it wouldn’t turn itself into Venus). Once you factor in libration a good deal of the planet’s day side and even some of it’s night side could be hospitable to life. The real issue is with how intense stellar flares will get – but with Gliese 581 this doesn’t seem to be a problem.

Ah, the media. Usually I do trust them, but the errors they can get are staggering, and this demonstrates they aren’t just errors in science. But hey, ain’t it sometimes fun to watch them “do it live?”

We just need to back off a little and let the scientific dudes make their cases.

There are literally hundreds of planet discoveries in the works and one more or less isn’t going to make much difference to the coming tidal wave (ahem, tsunami) of announcements. Nor are we going to send a probe to Gliese 581 anytime soon.

“manipulated data” might just mean ‘re-interpreted data’…. nothing malicious or underhanded. I say the zebra is white with black stripes, you see a black horse with white stripes…. which is right?
Just like the politicians claiming 180* of difference in status from a single set of facts, depending on their own bias…..

Don’t need no warp drive, just a REALLY big telescope and a good mask. And even the Federation now recognizes that warp drive damages spacetime or subspace or whatever. You surfer dudes really need to chill. Much easier to just lease a TARDIS. Then you can be there and back before dinner (yesterday).

Interesting write-up and discussion there BA, I’ve been wondering what your view of this was for ages! Thanks.

Difficult issue and clearly the media are playing the conflict up, warring scientists split into “hero” & “villain” camps heroes sells more papers than calm and reasoned debate between people disagreeing rationally over complex puzzles. Dare I say there are even faint echoes of “climategate” in the whole (non?/mis~?)use of the word “manipulate” there? In that perhaps the scientific phrasing is being misunderstood and played up by the media to create an illusion of a much worse situation and much more nasty personal clash than actually exists?

I hope the “Goldilock’s Planet” (Gliese 581g) exists but caution is required and we have to say we don’t know for sure until this dispute is resolved.

“This is fiendishly complex in practice …” Bayesian (though more probably actually Inverse Bayes) is fiendishly complex? Bayes’ Law and the inversion are among the simplest tools in statistical analysis. That is why I can laugh at most people who have a magical Bayesian Black Box – there is no need for magic or any appearance of it, the whole process is straightforward. It is disappointing that most people abuse the mathematics, possibly because they don’t even take the time to understand it, while others possibly abuse it on the assumption that many people believe it is a kind of magic and will accept an assertion as truth simply because “Bayesian” appeared in the title. Anyway, if Gregory did a proper analysis then there isn’t much hope of the planet turning out to be real. Either way, this should prompt people to develop a scheme for gathering data to rule out the existence of another planet. It also brings some attention to the difference in made-up confidence levels (I’m 90% sure that there’s really a planet – a claim pulled out of the ether) vs. claims supported by analysis. Nature is frequently counter-intuitive. I’ve just modeled a system as part of the instrument design process and the model results disagree with what I imagined reality would be. (Which was fortunate for me because in this instance it meant the instrument will be trivial to design.)

Does anyone have a list of all the scientific papers written on Gliese 581g? Not just the ones dealing with this scientific disagreement, but all of them, starting with the original announcement paper to the most recent…

The problem with a really BIG telescope is, it will amplify that suns light a lot, swamping the light of the much dimmer planet. Now, as soon as we develop methods to completely mask/negate the stellar light, what’s left over must be a planet.

Bose noise canceling headphones work for sound, so I see no reason we can’t develop something along those lines for EM.

(These limits to MY knowledge show why I are an engineer, instead of an astronomer,,,)

Like a hummingbird, I sip at the fountain of knowledge, while specialists chug.

@Nathan Hevenstone: useful resource is the SIMBAD service (quick Google search will find this), you can look up papers which refer to a given object. In my experience it is usually better to query for the star (GJ 581) than the individual planets. Unfortunately it does not list arXiv-only papers, but it does link to the ADS system. This allows you to query for any papers that cite a given paper, and the ADS does include the arXiv papers. If you are interested in the astronomical literature these services are very useful!

I mean you can take issue with ABC’s use of the word “manipulated” to characterize Vogt’s complaint, but in context it’s not completely unreasonable, and it is ultimately a characterization, a paraphrase if you will.

But then the Star says Vogt was quoted as saying “manipulated”, when that was not a quote!

It sounds like Gregory is saying that the data do not support the interpretation of the detection of 581g. He’s not actually saying the planet does not exist.

Remember, just because you don’t see it, does not mean it isn’t there. When you turn off all the lights in your room, the furniture doesn’t go away!

However, to take a reasonably famous analogous scenario, I have no evidence against there being a teapot orbiting the Sun between Earth and Mars, but that doesn’t mean I should believe it is there.

Is the existence of Gliese 581g a reasonable null hypothesis? Should we believe the planet is there until there is convincing evidence otherwise, despite there being no convincing evidence for its existence in the first place?

No. I think Gregory would agree (with both of us, right?) that “Gliese 581g does not exist” is an appropriate null hypothesis.

But I think he’s trying to point out that interpretation of the data is not sufficiently certain to allow us to trust *any* outcome *from the data* at this point. It’s more like he’s saying ‘the experiment cannot be considered finished’, and as such, the default outcome is ‘does not exist’.

But this ‘does not exist’ is because we accept of the null hypothesis, not because good data failed to reject it.

That was more my point about turning off the lights. Turning off the lights makes all your visual data into poor data. If your null hypothesis is “the room is empty”, visual data in a dark room will fail to reject that. But that’s because visual data in a dark room is poor data, not because the furniture went away.

Although none of the mainstream media would likely ever discuss it, this is as much about correct experimental design as it is about distant planets.

I have looked at the data, and analyzed it using regular, bayesian, inverse bayesian, and even reverse-inverse bayesian models, and the conclusion is irrefutable. The planet Earth does not exist, and never has. I would publish my results, but there are clearly no humans to read it. Plus, I don’t exist either, so I can’t write it.

Good grief Gregory, Vogt and Bad Astronomy! Let’s see the raw data and apply Bayesian analysis ourselves! There are so many able brains out there and not enough download links.

In fact, why not put this in the form of a video game so that laymen can contribute quality analysis through visual-spatial interaction and pattern recognition. I’d do it but I’m already working on a video game. Need to spin everyone else at the speed of light and buy some time.