About this Author

College chemistry, 1983

The 2002 Model

After 10 years of blogging. . .

Derek Lowe, an Arkansan by birth, got his BA from Hendrix College and his PhD in organic chemistry from Duke before spending time in Germany on a Humboldt Fellowship on his post-doc. He's worked for several major pharmaceutical companies since 1989 on drug discovery projects against schizophrenia, Alzheimer's, diabetes, osteoporosis and other diseases.
To contact Derek email him directly: derekb.lowe@gmail.com
Twitter: Dereklowe

February 21, 2013

An Incentive For Hype

Posted by Derek

Here's an article illustrating what goes into high-profile journal publications, and why you should always read past the title and the abstract. Björn Brembs noticed this paper coming out in Current Biology on fruit fly receptors and behavior, whose abstract claims that "blocking synaptic output from octopamine neurons inverts the valence assigned to CO2 and elicits an aversive response in flight". As Brembs puts it:

We currently have a few projects in our lab that target these octopamine neurons, so this was a potentially very important finding. It was my postdoc, Julien Colomb, who spotted the problem with this statement first. In fact, if it wasn't for Julien, I might have never looked at the data myself, as I know the technique and I know and trust the lab the paper was from. I probably would just have laid the contents of the abstract to my memory and cited the paper where appropriate, as the results confirmed our data and those in the literature (a clear case of confirmation bias on my part).

When you look harder, you find that yes, the genetically manipulated flied do seem averse to carbon dioxide plumes. But when you check the control experiments, you find that the two transgenes added to the flies (independent of the change to the octopamine system that's the subject of the paper) both decrease the tropism for CO2. So there's really no way of knowing what the effect of both of them might be, octopamine signaling or not, and you might well suspect that the two of them together could hose up the carbon dioxide response without invoking the receptor pathways at all.

As Brembs says, though, the authors aren't trying to hide this. It's in the body of their paper. Abstract be damned, the paper itself states:

"We note that the Tdc2-GAL4/+ driver line does not spend a significantly greater amount of time in the CO2 plume by comparison to air, but this line, as well as the UAS-TNT/+ parent line, spends significantly more time in the CO2 plume in comparison to their progeny. Therefore, this experimental result cannot be fully attributable to the genetic background."

No, not fully attributable at all, especially if the progeny show some sort of additive effect of the two transgenes. Of course, if you water down your conclusions too much, you might not get the paper into as good a journal as you'd like. I'll let Brembs sum up:

To make this unambiguously clear: I can't find any misconduct whatsoever in this paper, only clever marketing of the sort that occurs in almost every 'top-journal' paper these days and is definitely common practice. On the contrary, this is exactly the behavior incentivized by the current system, it's what the system demands, so this is what we get. It's precisely this kind of marketing we refer to in our manuscript, that is selected for in the current evolution of the scientific community. If you don't do it, you'll end up unemployed. It's what we do to stay alive.

If there's anyone out there who thinks that this doesn't go on in the chemistry literature, my advice is to please look around you a bit. This sort of thing goes on all the time, and I'd guess that most of us automatically dial down the statements in paper titles and abstracts as we read them, without even realizing any more that we're doing so. But in a case like this (and there are many others), even that process will still let erroneous conclusions into your head. And we all have enough of those already.

The editor should have never let this paper be published in its present form. It seems that the conclusions are not supported by the results. The same conclusions that cited in the abstract and title...

This is where the reviewer(s) and editor are clearly at fault. The data shows that their experiment was meaningless. The fact that they were (ultimately) honest in their description of the data is something that should only be expected.

One of the things which has disappointed me most about biomedical science is that the literature is completely unreliable. It is frankly not a good place to start on a drug discovery effort if you are going after anything relatively new.

At least these folks were honest in the end -- this is a higher level than most of the literature out there rises to.

I really think that this i becoming one of the major impediments to advancing our field: how can you stand on the shoulders of giants if the giants are mythological?

I'm leaving science partly because I don't want to play that game. Permanently lying about my work in order to do my work. I feel like just doing solid research will not keep you afloat in an academic environment. And if I have to sell out, at least I want to get paid!