Sub menu

Another paper disputes Bem’s psi findings but also serves as excellent instruction

In 2011, Daryl Bem conducted series of 9 experiments in which he claimed evidence for a precognition. The studies were widely reported in the press but strongly questioned by the scientific community. Soon after, attempts were made to reproduce them as methodological flaws were found. These attempts failed to show the same effect. Out of these results also arose a controversy of publishing the results of the duplication attempt. Now, another duplication by Galak, LeBoeuf, Nelson, and Simmons provides ADDITIONAL evidence that Bem’s initial results were flawed, that the effect size is essentially zero.

Steven Novella presents a commentary on the new results and why he feels this new paper is highly valuable to skeptics.

Bem’s studies have not fared well in replication. Earlier this year Ritchie, Wiseman, and French published three independent replications of Bem’s 9th study, all negative. Of further interest is that the journal that originally published Bem’s article had declined to publish Ritchie et al’s paper claiming that they don’t publish replications. This decision (and editorial policy) was widely criticized, as it reflects an undervaluing of replications.

It’s good to see that the journal has relented and agreed to publish a replication. Galak, LeBoeuf, Nelson, and Simmons should be commended, not only on their rigorous replication but their excellent article, which hits all the key points of this entire episode.

As we see, there were many problems with the methods and statistical analysis of Bem’s original paper. Bem argues that each problem was small and by itself would not have changed the results. This argument, however, misses a critical point, made very clear in another recent paper – one that was also cited and discussed in the Galak paper.

Simmons et al published a paper demonstrating how easy it is to achieve false positive results by exploiting (consciously or unconsciously) so-called “researcher degrees of freedom.”

The least important implication of the recent paper by Galak et al is that it provides further evidence against psi as a real phenomenon, and specifically against the claims of Daryl Bem. Psi is a highly implausible hypothesis that has already been sufficiently refuted, in my opinion, by prior research.

Source: Science Based Medicine

This is a technical piece that may be a bit difficult for non-scientists. But, as always, SBM is readable and gives invaluable insight into this complicated issue. The Bem results were highly controversial for several reasons. But this incident can serve as a wake up call for better research methodology and publication issues. And it illustrates the complexity of research and why we should be skeptical.

There IS a chance that evidence for psi can be worthy of attention but it’s going to take a LOT more than this one study (currently on very shaky ground) to convince those of us who understand WHY extraordinary claims require extraordinary evidence.