We’ve all had that experience of going purposefully from one hypothesis to another, only to get there and forget why we made the journey. Four years ago, researcher Daryl Bem and his colleagues stripped this effect down, showing that the simple act of obtaining a statistically significant comparison induces publication in a top journal. Now […]

Lee Sechrest writes: Here is a remarkable paper, not well known, by Paul Meehl. My research group is about to undertake a fresh discussion of it, which we do about every five or ten years. The paper is now more than a quarter of a century old but it is, I think, dramatically pertinent to […]

Yesterday we discussed the sad and disturbing career of psychology researcher Ellen Langer, who was was famous (to me) for her 1975 article on the illusion of control, “defined as an expectancy of a personal success probability inappropriately higher than the objective probability would warrant.” And then, in her own research, she herself became subject […]

It all started when Lee Sechrest pointed me to this post by James Coyne. Sechrest wrote: I know you have enough to do, and if you do not get to this…well, no problems. It is a blog by Jim Coyne taking apart a “classic” study in social psychology, originally published in the early ’70s. Implausible […]

From a recent news article by Laura Helmuth, I learned this amusing fact about DNA-discoverer James Watson: “he told a New York Times reporter 16 years ago that a researcher was ‘going to cure cancer in two years.’” Here’s the link to the NYT story, dated 3 May 1998: Within a year, if all goes […]

Following up on my discussion of Steven Pinker’s writing advice, Pinker and I had an email exchange that cleared up some issues and raised some new ones. In particular, Pinker made a connection between the difficulty of writing and some research findings in cognitive psychology. I think this connection is really cool—I’ve been thinking and […]

Jonathan Falk sends a message with the heading, “Garden of forking paths, p value abuse, questionable causality, you name it,” this link to an article in JAMA Internal Medicine, and the following remarks: Unfortunately, I can only see the first page of this article, but it seems to contain all the usual suspects. (a) Forking […]

Noon Thursday, January 29, 2015, in William James Hall 765 room 1: The Statistical Crisis in Science Andrew Gelman, Dept of Statistics and Dept of Political Science, Columbia University Top journals in psychology routinely publish ridiculous, scientifically implausible claims, justified based on “p < 0.05.” And this in turn calls into question all sorts of […]

In a recent discussion of replication in science (particularly psychology experiments), the question came up of how to interpret things when a preregistered replication reaches a conclusion different from the original study. Typically the original, published result is large and statistically significant, and the estimate from the replication is small and not statistically significant. One […]

In the context of a listserv discussion about replication in psychology experiments, someone wrote: The current best estimate of the effect size is somewhere in between the original study and the replication’s reported value. This conciliatory, split-the-difference statement sounds reasonable, and it might well represent good politics in the context of a war over replications—but […]

Alan Sokal writes: We know perfectly well that our politicians (or at least some of them) lie to us; we take it for granted; we are inured to it. And that may be precisely the problem. Perhaps we have become so inured to political lies — so hard-headedly cynical — that we have lost our […]

Under the heading, “Results too good to be true,” Lee Sechrest points me to this discussion by “Neuroskeptic” of a discussion by psychology researcher Greg Francis of a published (and publicized) claim by biologists Brian Dias and Kerry Ressler that “Parental olfactory experience [in mice] influences behavior and neural structure in subsequent generations.” That’s a […]

I (Bob) spent last weekend at Biosphere 2, collaborating with soil carbon biogeochemists on a “super model.” Model combination and expansion The biogeochemists (three sciences in one!) have developed hundreds of competing models and the goal of the workshop was to kick off some projects on putting some of them together intos wholes that are […]

I don’t know what ATR is but I’m glad somebody is on the job of prohibiting replication catastrophe: Seriously, though, I’m on a list regarding a reproducibility project, and someone forwarded along this blog by psychology researcher Simone Schnall, whose attitudes we discussed several months ago in the context of some controversies about attempted replications […]

This post is by Phil Price. I’m posting it on Andrew’s blog without knowing exactly where he stands on this so it’s especially important for readers to note that this post is NOT BY ANDREW! Last week a prominent scientist, representing his entire team of researchers, appeared in widely distributed television interviews wearing a shirt […]

Here I am one day: Let me conclude with a statistical point. Sometimes researchers want to play it safe by using traditional methods — most notoriously, in that recent note by Michael Link, president of the American Association of Public Opinion Research, arguing against non-probability sampling on the (unsupported) grounds that such methods have “little […]

I agree with Deborah Mayo who agrees with Carlo Rovelli that “Science does not advance by guessing. It advances by new data or by a deep investigation of the content and the apparent contradictions of previous empirically successful theories.” And, speaking as a statistician and statistical educator, I think there’s a big problem with the […]

Deborah Mayo and I had a recent blog discussion that I think might be of general interest so I’m reproducing some of it here. The general issue is how we think about research hypotheses and statistical evidence. Following Popper etc., I see two basic paradigms: Confirmationist: You gather data and look for evidence in support […]

Meritocracy won’t happen: the problem’s with the ‘ocracy’ Does the sex of your child affect your political attitudes? More hype about political attitudes and neuroscience Modern polling needs innovation, not traditionalism Who cares about copycat pollsters? The mythical swing voter Mythical swing voter update No, all Americans are not created equal when it comes to […]