Skeptophilia (skep-to-fil-i-a) (n.) - the love of logical thought, skepticism, and thinking critically. Being an exploration of the applications of skeptical thinking to the world at large, with periodic excursions into linguistics, music, politics, cryptozoology, and why people keep seeing the face of Jesus on grilled cheese sandwiches.

Wednesday, March 8, 2017

Contradicting the narrative

In her marvelous TED Talk "On Being Wrong," writer and journalist Kathryn Schulz describes a "series of unfortunate assumptions" that we tend to make when we find out that there are people who disagree with us.

First, we tend to assume that the dissenters are simply ignorant -- that they don't have access to the same facts as we do, and that if we graciously enlighten them, they'll say, "Oh, of course!" and join our side. If that doesn't work, if the people who disagree with us turn out to have access to (and understand) the same facts as we have, then we turn to a second assumption -- that they're stupid. They're taking all of the evidence, putting it together, and are too dumb to do it right.

If that doesn't work -- if our intellectual opponents have the same facts as we do, and turn out to be smart enough, but they still disagree with us -- we move on to a third, and worse, assumption; that they're actually malevolent. They have the facts, know how to put them together, and are suppressing the right conclusion (i.e. ours) for their own evil purposes.

This, Schulz says, is a catastrophe. "This attachment to our own sense of rightness keeps us from preventing mistakes when we absolutely need to," she says, "and it causes us to treat each other terribly."

I got a nice, and scientific, object lesson in support of Schulz's claim yesterday, when I stumbled across a paper in PLoS One by Clinton Sanchez, Brian Sundermeier, Kenneth Gray, and Robert J. Calin-Jagemann called "Direct Replication of Gervais & Norenzayan (2012): No Evidence That Analytic Thinking Decreases Religious Belief." Apparently five years ago, a pair of psychological researchers, Will Gervais of the University of Kentucky and Ara Norenzayan of the University of British Columbia, had published a study showing that there was an inverse correlation between religious belief and analytical thinking, and further, stimulating analytical thinking in the religious has the effect of weakening their beliefs.

[image courtesy of Lucien leGray and the Wikimedia Commons]

Well, all of that fits nicely into the narrative we atheists would like to believe, doesn't it? Oh, those religious folks -- if we could just teach 'em how to think, the scales would fall from their eyes, and (in Schulz's words) they'd "come on over to our team." The problem is, when Sanchez et al. tried to replicate Gervais and Norenzayan's findings, they were unable to do so -- despite (or perhaps because of) the fact that Sanchez et al. used a much larger sample size and tighter controls. The authors write:

What might explain the notable difference between our results and those reported by G&N? We can rule out substantive differences in materials and procedures, as these were essentially identical. We can also rule out idiosyncrasies in participant pools, as we collected diverse samples and used extensive quality controls. Finally, we can also rule out researcher incompetence, as we were able to detect an expected effect of similar size using a positive control.

One possibility is that Study 2 of G&N substantially over-estimated the effect of the manipulation on religious belief. This seems likely, not only because of the data presented here but also because evidence published while this project was in progress suggests that the experimental manipulation may not actually influence analytic thinking...

Based on our results and the notable issues of construct validity that have emerged we conclude that the experiments reported by G&N do not provide strong evidence that analytic thinking causes a reduction in religious belief. This conclusion is further supported by results from an independent set of conceptual replications that was recently published which also found little to no effect of analytic thinking manipulations on religious belief.

To their credit, Gervais and Norenzayan not only cooperated with the research of Sanchez et al., they admitted afterwards that their original experiments had led to a faulty conclusion. In fact, in his blog, Gervais gives a wryly humorous take on their comeuppance, by presenting the criticisms of Sanchez et al. and only at the end revealing that it was his own paper that had been, more or less, cut to ribbons. He says of Sanchez's team, "I congratulate them on their fine work."

He also included the following in his postscript:

FFFFFFFFFFUUUUUUUUUUUUUCCCCCCCCCKKKKKKKKKKKK!!!!

Understandably. As Schulz points out, it's often devastating and embarrassing to find out that we screwed up. Doubly so when you're a scientist, since your reputation and your livelihood depend on getting things right. So kudos to Gervais and Norenzayan for admitting their paper hadn't shown what they said it did.

So not only is this a great example of science done right, in the larger analysis, it tells us atheists that we can't get away with dismissing religious folks as simply not being as smart and analytical as we are. Which, honestly, is just as well, because it would leave me trying to explain friends of mine who are honest, smart, well-read, logical... and highly religious. It supports the kinder (and more accurate) conclusion that we're all trying to figure things out as best we can with what information we have at hand, and the fact that we come to radically different answers is testimony to the difficulty of understanding a complex and fascinatingly weird universe with our limited perceptions and fallible minds.

Or, as Schulz concludes, "I want to convince you that it is possible to step outside [the feeling of being right about everything], and that if you do so it is the greatest moral, intellectual, and creative leap you can take... If you really want to rediscover wonder, you need to step out of that tiny, terrified space of rightness, and look around at each other, and look out at the vastness and complexity and mystery of the universe, and be able to say, 'Wow. I don't know. Maybe I'm wrong.'"