Scanning Dead Salmon in fMRI Machine Highlights Risk of Red Herrings

Neuroscientist Craig Bennett purchased a whole Atlantic salmon, took it to a lab at Dartmouth, and put it into an fMRI machine used to study the brain. The beautiful fish was to be the lab’s test object as they worked out some new methods.

So, as the fish sat in the scanner, they showed it “a series of photographs depicting human individuals in social situations.” To maintain the rigor of the protocol (and perhaps because it was hilarious), the salmon, just like a human test subject, “was asked to determine what emotion the individual in the photo must have been experiencing.”

If that were all that had occurred, the salmon scanning would simply live on in Dartmouth lore as a “crowning achievement in terms of ridiculous objects to scan.” But the fish had a surprise in store. When they got around to analyzing the voxel (think: 3-D or “volumetric” pixel) data, the voxels representing the area where the salmon’s tiny brain sat showed evidence of activity. In the fMRI scan, it looked like the dead salmon was actually thinking about the pictures it had been shown.

“By complete, random chance, we found some voxels that were significant that just happened to be in the fish’s brain,” Bennett said. “And if I were a ridiculous researcher, I’d say, ‘A dead salmon perceiving humans can tell their emotional state.'”

The result is completely nuts — but that’s actually exactly the point. Bennett, who is now a post-doc at the University of California, Santa Barbara, and his adviser, George Wolford, wrote up the work as a warning about the dangers of false positives in fMRI data. They wanted to call attention to ways the field could improve its statistical methods.

Researchers get up to 130,000 voxels from each set of scans they do of a brain. They have to comb all that data for signals that indicate something is happening in a particular region of the brain. The fMRI data has a lot of natural noise, though, and with the amounts of data generated in the work, chance can play some tricks. Bennett compared the fMRI data problems to a particularly strange kind of darts game.

“In fMRI, you have 160,000 darts, and so just by random chance, by the noise that’s inherent in the fMRI data, you’re going to have some of those darts hit a bull’s-eye by accident,” he said.

Like a sophisticated version of Photoshopping the contrast on a photograph, neuroscientists can filter the fMRI data to highlight the signal within the noise, but in so doing, rigorous statistical checks have to be maintained.

“We could set our threshold so high that we have no false positives, but we have no legitimate results,” Bennett said. “We could also set it so low that we end up getting voxels in the fish’s brain. It’s the fine line that we walk.”

Bennett’s point is that a suite of methods known as multiple comparisons correction can allow researchers to maintain most of their statistical power while keeping the danger of false positives at bay.

The work highlights that brain science is highly data-driven and statistical now. Although the visualizations — usually some orangey spots on an otherwise dark brain scan — seem simple, the data collection and interpretation that go into producing them is intense.

The point of the salmon study isn’t to prove that fMRI shouldn’t be used or is worthless. Brain scientists can do things with fMRI machines they otherwise couldn’t, said Ed Vul, an MIT neuroscience graduate.

“Instead of hoping to find a particular [brain] lesion or getting monkeys and making the lesions, this method is much cheaper and easier,” Vul said, “and has allowed a much broader range of questions to be asked about the brain.”

Bennett’s paper has been turned down by several publications, but a poster on the work received an appreciative audience at the Human Brain Mapping conference earlier this summer. Neuroscience researchers have been forwarding it to each other for weeks.

Perhaps the dash of humor helps the medicine go down. And using a tasty, delicious fish instead of a human to make the false-positives point has an ancillary benefit: After you’re done with the research, it can be reused via culinary post-processing.

“Any good scientist would want to know the details of post-scan culinary post-processing of the subject and the hedonic results of degustation of the subject when studying a population of salmon, even with N=1,” wrote David Perlman, a neuroscience graduate student at the University of Wisconsin, in a comment on Bennett’s blog. “I would be very appreciative if you would make this information publicly available. Thanks!”

Here’s The Thing With Ad Blockers

We get it: Ads aren’t what you’re here for. But ads help us keep the lights on. So, add us to your ad blocker’s whitelist or pay $1 per week for an ad-free version of WIRED. Either way, you are supporting our journalism. We’d really appreciate it.