Falsehood perishes… eventually

The damage that falsehood can do, if unchallenged and/or perpetuated over a period of time, can be considerable, often long lasting, both to individuals (for whom it is typically permanent — ask anyone wrongly convicted of a crime that they did not commit) and to humankind as a whole. For this reason, it seems a good idea to have procedures in place not only to spot it, but also to expose and debunk falsehood swiftly and effectively, before it spreads.
There exist circumstances in which falsehood acquires a pernicious resilience, even in the absence of a concerted effort on the part of anyone to preserve it. All that is needed is a sufficiently robust system of perverse incentives, which may come about for whatever reason and prove surprisingly hard to die.

Thought control, propaganda, conspiracy theory etc.
One of the favourite propaganda techniques on the part of corporate mass media, one to which conspiracy theorists like to point, consists of news outlets giving great emphasis to stories that bolster the agenda of those who control the media themselves.
These stories make national headlines, are the subject of investigations, receive in-depth coverage, and are aired during prime time. To them politicians refer, in order to promote their pet causes. Most of the time the stories are accurate, but in some cases, they turn out to be bogus. At that point, of course, corrections and/or retractions are in order. But often times there is lack of interest, or even outright reluctance on the part of the news outlets to give retractions and corrections the same prominence that was given to the stories that prompted them, precisely because of the perceived ideological value of the latter.
Thus, in many cases retractions and corrections are… well, not quite “swept under the rug”, but may be printed on the last page and/or in some obscure section of the newspapers that carried the original story on the front page, or broadcast late at night, to much more limited readerships and audiences [0]. Not surprisingly, the initial (inaccurate) story retains some credence among significant strata of the population, even long after being officially recanted and discredited.
One does not have to invoke “conspiracy”, though. The above scenario is actually quite common; in fact, I do not believe that there exists a single human activity immune from the above dynamics. It may take place in very different contexts, even when no single, powerful and influential individual, group thereof or organization can be identified, aiming at imposing a given “party line”.

Of course, this would not happen in the sciences…
For all its pretence of rigour and objectivity; claims notwithstanding from its practitioners of an unwavering commitment to the pursuit of Truth; despite our collective confidence (often bordering on naiveté) in its built-in mechanisms of identification and removal of falsehood through carefully controlled reproducibility; science remains ultimately a human enterprise, hence not immune from the same dynamics that affect other activities.

Case in point: dissemination of scientific findings takes place through publication on specialized journals, appropriate for a given discipline or field of inquiry. Of course, not all journals are created equal. There exist flagship publications — high profile, “glamourous” journals (GJ). Albeit to different degrees, publishing in a GJ ensures visibility and citations, and confers to the work and its authors authoritative prestige.
So, for example, publication in, say, Nature, is the scientific article equivalent of a news story being featured on the front page of the city paper, or during a prime time television newscast. Publication in a more specialized journal, possibly respected but narrower in scope and with a more limited readership, roughly corresponds to running the same news story late at night, right before the informercials or some rerun of “Cheers”, or placing it in some section of the paper of lesser interest, together with the coupons and Sunday’s specials.
In both cases the news story is published/aired, but the impact can be vastly different [1].

Oops…
It is inevitable that, every once in a while, in spite of the scrutiny to which are subjected articles submitted for publication to scientific journals (the more prestigious the journal, the more severe the scrutiny — at least in principle), a paper containing incomplete, erroneous, or otherwise inaccurate results and/or conclusions will appear in print, sometimes even in a GJ.
Now, why and how that happens is an interesting but very complex subject, not one that I am keen on discussing now — suffice to say that people publish incorrect results for many reasons.
The good news is, science is self-correcting. Others will repeat the experiments or calculations and prove them wrong if they are. They will publish their correct results, set everyone straight, the author(s) of the original work will either begrudgingly acknowledge the error or look silly, and some progress will have been accomplished anyway.
So, in the long run, things will be sorted out — that is really not the issue.

The issue is: what does “in the long run” mean [2] ? How long should or will it take for the community of interested scientists, especially those directly affected by the error (i.e., working in the same or a related field), to become broadly aware that an inaccurate conclusion was originally published, and that that article should perhaps be disregarded ? How long can the community afford “falsehood” to be held as “truth” by a substantial fraction of investigators ? And what actions, if any, should be taken to make that time as short as possible ?

Prime time or late night retraction ?
Let us be concrete and suppose that a group of investigators (say, group A) publishes an article on some GJ, describing important findings — nothing short of a fundamental discovery. This article sparks interest and activity among many other investigators; the authors of A receive prizes, accolades, citations and (last but not least) substantial funds to pursue further research on that subject.
Suppose now that a second, independent group (B) of investigators, attempt to reproduce the findings of A, failing consistently. After some time, group B’s investigators identify a procedural flaw in the original work of group A, a subtle effect overlooked by group A, that caused what appeared to be a genuine new phenomenon. Group B demonstrates that such an observation is spurious — nothing but an artifact. No new phenomenon has in fact been observed. Group B writes a paper, thoroughly debunking the findings reported by A.Question:Where should the paper by group B be published ?

Duh…
Isn’t the answer to the above question obvious ? Why, GJ, of course ! Isn’t it a no-brainer, really ?
Would you not expect group B to submit its article to GJ ? And, if you were the Editor of GJ, and became convinced that the work by group A is erroneous, would you not want to make damn sure that the first piece of work correcting it also be published in your journal ?
Would you not take that to heart, regard it as a necessary step to ensure the continued respectability and prestige of your publication ? What kind of rationale could the Editor of GJ offer for declining publication of an article demonstrably, convincingly correcting or revising work already published in that same journal ?
Can you imagine the Editor of GJ arguing against publication of the work by group B, suggesting instead that it be submitted to a “specialized journal”, where it will not be read as widely, based on one or more of the following arguments [3]:

There is nothing intrinsically interesting in “disproving” anything. The paper by B reports nothing fundamentally new or exciting, and GJ is exclusively about fundamental or exciting new things.

Unfortunate as that is, the fact that a flawed article was published is no reason to follow up with a “boring” (if correct) one.

Believe it or not, the above describes as accurately as I can, what I have seen happening of late to a number of individuals playing the role of the hypothetical group B, i.e., submitting a paper to GJ debunking results by others previously published in that same journal. On one occasion, I was in that situation myself, on the receiving end of the very arguments above.
The end result is exactly the same as in the case of media propaganda. Incorrect beliefs and/or inaccurate information, especially on potentially important subjects, are cited, discussed and retain credibility among many, for a long time, as if they were true, even after being proven incorrect. This happens almost solely by virtue of the venue in which they were first published. Most investigators will not bother to look for any subsequent work casting doubts on those contentions, if that work does not receive the same “prime time exposure” of the original article, ending up published in a “specialized journal” [4].
This is not a desirable state of affairs, in my opinion. Something should be done to curb the above dynamics.

In praise of party poopers
Don’t get me wrong, I like exciting, ground breaking discoveries as much as the next scientist. However, I still believe that “Extraordinary claims require extraordinary evidence“.
What I do not like is a system in place that not only provides incentives for investigators to forgo thoroughness in favor of publicity, but also makes it possible and likely for claims based on flimsy evidence to remain the official truth for years.
A solid piece of research work debunking, or casting serious doubts on outstanding, important claims published on a high profile venue should by definition be published on the same venue. Relegating it to a lesser venue does a huge disservice to science as a whole, at least in two ways.

First off, it keeps cogent information away from researchers. Lack of such information has a number of unfortunate consequences, for example it encourages the continuation of research efforts based on fundamentally flawed premises.
But there is also another, potentially even more deleterious consequence of denying prime time exposure to scientific “news” that throw cold water on the euphoria generated by some hasty, imprudent claim. Researchers who check/attempt to reproduce/confirm independently results of calculations or experiments done by others, especially those calculations or experiments that appear to have yielded potentially significant, unexpected or interesting results, perform a crucial, vital, all-important task.
A scientific establishment that de facto makes pariahs of these individuals, by downplaying and giving minor recognition to their contribution, is broken.
How do we motivate a researcher (especially a junior one) to pursue the all-important exercise of repeating someone else’s study, if the perception becomes widespread that the payoff in terms of recognition will be low ? Knowing from from the start that, no matter what the outcome of the experiment or calculation, the end result will be an article on a relatively minor journal (either because the original “interesting” results are confirmed, in which case the calculation is regarded as “mere confirmation”, or because they are proven wrong and a potentially “exotic” scenario is replaced by a more conventional, “boring” one), constitutes a powerful disincentive. The long-term consequence may be that of promoting, particularly among my younger colleagues, the frantic production of low-quality, sloppy works, whose main/only merit may be that of proposing some “glamorous” (if dubious) scenario, at the expense of more rigorous studies whose outcome may be more sober, more “conventional”, but accurate.

Notes

[0] Strictly speaking, the news organizations are doing what they are supposed to do. Hidden as they might be, published retractions can nonetheless be regarded as proof of objectiveness; and indeed, anyone willing and able to read the paper down to the last page and/or stay up late at night (i.e., anyone with a “sufficiently powerful microscope” as Noam Chomsky would put it), would become aware of the shakiness of the initial story, and consequently could exercise skepticism, re-evaluate it, and possibly dismiss it altogether. Clearly, however, the vast majority of those who heard the original version will end up holding it as “true”, perpetuate it and disseminate it for a much longer time, not having seen it officially recanted, or even called into question.

[1] We are now approaching a point where the Litmus test of the importance and influence of a scientific result, or even of a whole field of study, is whether it makes for “Nature material”. Contributions published in journals that are discipline-specific (including highly respected ones) are increasingly branded as “good but not quite exceptional“, especially by university administrators, with a subtle but real pejorative slant. This is of course a problem in and of itself, but not one that I am discussing in this post.

[3] I would like to stress that I am talking about a situation in which the work of group B has removed any doubt regarding the incorrectness of the claim of group A.

[4] Of course, the arguments against publications of the paper by group B that I describe here are self-serving and disingenuous. In practice, the reluctance on the part of the Editor may well be attributable to one of more of the following factors:

Displeasure with seeing an exciting scenario disproven.

Unwillingness to do what essentially amounts to admitting a mistake (publication of original paper by group A), which may negatively affect GJ’s reputation.

Genuine lack of interest for a paper which, correct as it might be, will likely not receive many citations (as most articles disproving anything).
Lest anyone out there think that GJs have a responsibility toward the scientific community, it may not be amiss to remind ourselves that the ultimate goal of the most prestigious GJs is making money, not dissemination of scientific results and knowledge. I can see someone having to make the choice between being the Editor of “correct but possibly boring” or “sexy but possibly wrong” results, not choosing what would best serve scientific progress.

Share this:

Like this:

Related

This entry was posted on June 2, 2012 at 1:35 am and is filed under Physics, Policy, Research, Science. You can follow any responses to this entry through the RSS 2.0 feed.
You can skip to the end and leave a response. Pinging is currently not allowed.

6 Responses to “Falsehood perishes… eventually”

See, this is one reason why I argue that one should not bash PRL too badly. Yes, it and its editors are often obnoxious and we all often wish there were more alternatives. But it is far more likely to at least publish a one-page comment. Which is not quite what you argue should happen, but which is more than NPG journals will easily do, where the equivalent to comments is much rarer. So, if there is a monopoly on “important papers”, I’d prefer that monopoly to be under the control of the APS, where I am a member, rather than under that of NPG, which has only itself to be accountable to.

I agree with you that science should be more open towards refutals as well as criticisms.

Instead of deeming critiques and refutals as counter productive, it should be viewed as a constructive act that either raises an important question or points out an important flaw. Where both of these results serve as an important foundation for progression by pointing out an area in need of improvement and revision.
It is the concealment of fasehood that is counter productive, not the act of critiquing and refuting.

How is it not interesting and exciting to disprove something? Quite to the contrary, it is the disproof that provides you with the oppurtunity to conduct further researsch and assert what is in fact the case. It is the dispoof that makes room for science. What’s so interesting about a truth that only pretends to be true? What’s so interesting about falsehood that’s left uncorrected?

It is rather dangerous and unethical to conceal a falsehood. Since science is responsible for knowledge, to provide people with the way things are, scientists should take a greater responsibity for their own words. The words that many will take for granted.

Scientists have moral obligation to correct itself, to maintain the truth and integrity of the things they assert.

Do scientists really have no control over the knowledge they establish? As to what is consider as important, unimportant, right, or wrong. That is, are those profit motivated journals the only means of effective communication in the scientific community? The profit motivate journals that isn’t even controlled by the people they try to represent. I don’t see why the scientists couldn’t find other ways to announce their errors….