The figure above illustrates the singing state (control condition) in(a) and the glossolalia state in (b). The authors say there is decreased regional cerebral blood flow (rCBF) bilaterally in the frontal lobes and unilaterally in the left caudate during glossolalia, so we'll just have to take their word for it. BUT:

Our results were hypothesis driven so comparisons were only tested for the major structures of the frontal, temporal, and parietal lobes, as well as the amygdala, hippocampus, striatum, and thalamus, and thus a correction for multiple comparisons was not performed.

[I don't think that having a hypothesis exonerates one from correcting for multiple comparisons.]

So should we be skeptical of these pretty pictures? Yes! The press routinely portrays fMRI as capable of mind reading, for example, and lie detection. However, Aguirre (an expert in fMRI methodology) says this is “science fiction, science fantasy.” What are his reasons for skepticism? FaithWorld outlines them in Beware brain scientists bearing gifts (gee-whiz journalists too…):

scientific awesomeness — “This is an incredible technology. Neuroimaging is not phrenology. It really is a scientific discipline that has reproducible results that makes valuable predictions that explain larges areas of cognition and cognitive neuroscience that previously had been inaccessible.”

image properties — “There’s definitely an esthetic in the presentation of this data. People see this as a natural aspect of the brain, not the result of tests. Some groups made a very wise investment in the display technology for how neuroimaging results were reported. Those were the images that got displayed on the covers of the top scientific journals and made a splash.”

thresholding — The brain images leave out data outside the main focus. “This contributes to the overly localised view of brain function. So we say, ‘ah this is the spot for love’ or whatever, because it’s all that we see.”

overinference — “It’s very easy to believe a lot of things about these images that might not be true… It’s also implied that when you’ve found activisation in a region, you’ve found the region ‘for’ something. But what does that mean?”

chicken versus egg problem — “Just because you find a difference between groups in some brain imaging measure does not mean that structural difference was genetically determined.” But the brain also develops according to its owner’s environment and experience, so this is too narrow a focus.

lurking Cartesian dualism — “In the way we think about people’s actions and describe the effect of diseases or drugs, there is frequently a lurking dualism there. We say, ‘oh it wasn’t his fault, his brain did that.’ Well, who else could it have been? Where else could those thoughts and feeling or plans have come from, except in the brain? This idea that the brain and the mind are separate is part of what makes these images so remarkable. Wow look! Here’s a part of the brain that’s more active when you’re feeling romantic love or not! That’s just astounding to folks who would have thought romantic love was outside the brain, in the heart or the soul and far away.”

illusion of inferential proximity — “It doesn’t automatically follow that a brain imaging technology is going to give you greater inferential leverage on a question than just talking to somebody. There’s an illusion that somehow you’re getting much closer to the behavior you want to measure, just because you’re measuring a brain image. That might not be the case.”

ease of imaging — Many hospitals have brain scanners and researchers can use them and free imaging software to create impressive images. “If you have an internet connection and a scanner, you can be a cognitive neuroscientist and publish a paper. Lots of the variance in the lousy scientific papers over these years can be explained this way. What will come out will be a well-formed brain image that will give the impression you must be a very good scientist because you created something that looks very polished.”

Explanations of psychological phenomena seem to generate more public interest when they contain neuroscientific information. Even irrelevant neuroscience information in an explanation of a psychological phenomenon may interfere with people's abilities to critically consider the underlying logic of this explanation. We tested this hypothesis by giving naïve adults, students in a neuroscience course, and neuroscience experts brief descriptions of psychological phenomena followed by one of four types of explanation, according to a 2 (good explanation vs. bad explanation) x 2 (without neuroscience vs. with neuroscience) design. Crucially, the neuroscience information was irrelevant to the logic of the explanation, as confirmed by the expert subjects. Subjects in all three groups judged good explanations as more satisfying than bad ones. But subjects in the two nonexpert groups additionally judged that explanations with logically irrelevant neuroscience information were more satisfying than explanations without. The neuroscience information had a particularly striking effect on nonexperts' judgments of bad explanations, masking otherwise salient problems in these explanations.

6 Comments:

These are all good points. But one can make pretty and misleading pictures in many fields (ERPs, single unit recordings, psychophysics etc). In addition, fluffier fields that are harder to visualize (take social psychology) have the equivalent problem with "pretty stories" that are in large part illusory explanations. Uncritical popular media, academic institutions that care more about how popular something is because it tends to translate into dollars, lower standards for awarding Ph.Ds; these are all factors that contribute to this problem. I see them at Boston College (where I've been for the last 20 years) all the time. It's not a problem limited to cognitive neuroscience, in my opinion.

You've got a point there. It would seem that impressive pictures and anecdotes carry more weight of conviction than logic or evidence. That, if nothing else, is why scientific education is important for anyone.

By the way, I've been following this blog with a lay interest for a while (I'm a maths student). I always enjoy your posts, but rarely have anything to add.

Anonymous is right, spurious results are not limited to fMRI, but the problem is exacerbated by the sheer volume of data and the fact that people do whole-brain analyses with inadequate or no correction for multiple comparisons. The visual appeal of fMRI doesn't help either.

Thanks for your description of my presentation, and for posting the photoshop artistry of my former grad student, Daniel Drucker.

I also wanted to ensure that the relationship between neuroimaging and "inferential distance" be properly attributed to Adina Roskies, who has thought deeply and written eloquently on the topic [Roskies, A.L. (2008) “Neuroimaging and inferential distance.” Neuroethics, 1: 19-30.]

Links to this post:

About Me

Born in West Virginia in 1980, The Neurocritic embarked upon a roadtrip across America at the age of thirteen with his mother. She abandoned him when they reached San Francisco and The Neurocritic descended into a spiral of drug abuse and prostitution. At fifteen, The Neurocritic's psychiatrist encouraged him to start writing as a form of therapy.