A personal weblog of developmental cognitive neuroscience.

The Story Behind the Atlantic Salmon

The Atlantic Salmon fMRI poster has garnered a fair amount of attention since its presentation at the Human Brain Mapping conference last June in San Francisco. So far the reaction from other researchers has been almost unanimously positive. A sizable number of people stopped by the poster while it was displayed and Rainer Goebel (of BrainVoyager fame) was kind enough to give the fish a shout-out during the closing ceremonies (see photo).

All in all I am quite pleased that the Salmon seems to be generating a fresh discussion of multiple comparisons correction in neuroimaging.But, how did it all begin? I mean, really, why would anybody want to scan a fish? This was one of the top five questions I was asked during the HBM poster session. It it a story that deserves to be told, and a weblog post is perhaps the ideal medium to tell it. So, for all readers who are curious, I have written up the story of the Salmon.

The story begins during my first year in graduate school at Dartmouth College. I was working with Abigail Baird on fMRI studies investigating the maturation of decision-making and we were developing a large number of new MRI protocols to use with adolescents and adults. Not wanting to waste valuable magnet time imaging and reimaging a MRI phantom, we instead challenged ourselves to scan the most curious objects we could find at the local grocery store.

For our first attempt we scanned a pumpkin. One result of this endeavor can be seen here. This is a pretty standard fruit to scan, as just about every imaging center around the country obtains a T1-weighted image of them in late October. Still, it was exciting to us. During the next pilot testing session Abby brought in a Cornish game hen to be scanned. This really upped the ante, as we had now put a dead bird into the head coil. When pondering our next step the comment was made: “we should scan a whole fish”.

I picked up the salmon from our local supermarket early on an early Saturday morning in spring of 2005. The clerk behind the counter was a little shocked to be selling a full-length Atlantic salmon at 6:30 AM, especially when I told her what was about it happen to it. About an hour later we were in the imaging center with the fish wrapped in plastic and securely placed within the head coil. We proceeded to test our entire protocol with the salmon in the magnet. In total, we did an anatomical localizer scan, four functional runs, a T1-weighted anatomical scan, and a diffusion tensor imaging (DTI) scan.

After transferring the data off of the scanner we first took a look at the high resolution anatomical image. It was simply incredible. Slice the fish along the sagittal plane and you could see the fish split right down the middle. Slice the fish coronally and you could see what looked like salmon steaks on the viewer. By far it was our crowning achievement in terms of ridiculous objects to scan. Then, our curiosity satisfied, I socked the salmon data away for the next three years.

In early 2008 I was working with my co-adviser George Wolford on a presentation he was giving regarding the multiple comparisons problem in fMRI. We were discussing false positives in MRI phantom data and I brought up the idea of processing the salmon fMRI data to look for some ‘active’ voxels. I ran the fish data through my SPM processing pipelines and couldn’t believe what I saw. Sure, there were some false positives. Just about any volume with 65,000 voxels is going to have some false positives with uncorrected statistics. Rather, it was where the false positives occurred that really floored me. A cluster of three significant voxels were arranged together right along the midline of the salmon’s brain. If they would have been anywhere else the salmon would have been just a curious anecdote, but now we had a story.

George presented the salmon data at our local fMRI methods group, but nothing much happened for a while after that. George was convinced that we could/should publish the data and that it was an excellent example of the multiple comparisons problem. I was less convinced, remarking about how silly that would be and how terrible it would be for a young postdoc to become known as ‘the fish guy’. For the next year we went back and forth about the issue, until one day in January, 2009. George was out in Los Angeles and came up to UCSB to visit. Over lunch he said that it was time to ‘get the fish out’. I relented, and agreed to start writing the paper.

About a week later the HBM conference poster deadline came around and we decided to submit the salmon as an abstract. We genuinely wanted it to be a part of the conference, but we really doubted that it would be approved. How right we were. Through some sources close to the matter I have learned that the salmon poster was indeed rejected by every reviewer who saw the abstract. Just about everyone thought it was a joke – some rogue student who was playing a prank on the OHBM. It was only when the rejected abstract went before the OHBM Program Committee that it was given approval to stay as part of the conference. I hear that even that vote was contentious.

While the abstract reviewers were busy rejecting the salmon poster my co-authors and I were diligently writing a full-on salmon manuscript. The overall outline of the paper had been in our heads for some time and the writing went rather quickly. By April we had a polished manuscript ready for review and we sent it off to a major neuroimaging journal. Within a week we heard back that it was being rejected on an editorial basis. We heard that there were several major discussions within the journal staff regarding whether to even review the piece. In the end they decided to pass the responsibility, and the trouble, on to another journal.

That brings us to today. The ‘Post-Mortem Atlantic Salmon’ was a strong success at the OHBM conference. It is also under review at a second major neuroimaging journal. The more I think about the affair the more I believe that the fish has the chance to impact the field of neuroimaging in a very positive way. Predefined significance thresholds with a specified cluster extent are a weak control to the problem of false positives in imaging data. Statisticians and methods researchers have argued about the need for multiple comparisons correction for some time. In just one figure the salmon data illustrates exactly why we need stronger controls for the false positive problem in fMRI. I hope it finds a good home in an open-minded journal.

You can find a copy of the ‘Post-Mortem Atlantic Salmon’ poster at this link:

[...] author Craig Bennett explains further on his blog: In early 2008 I was working with my co-adviser George Wolford on a presentation he was giving [...]

Neil - September 18th, 2009

I have to say, I can understand why reviewers have rejected it; you really should add data obtained in dead tuna, sea bass, and squid to make these findings generalizable. One simply can’t say, based on your salmon data, that the fundamental properties of statistical comparison will apply in all species of dead sea creature.

[...] the salmon scanning would simply live on in Dartmouth lore as a “crowning achievement in terms of ridiculous objects to scan.” But the fish had a surprise in store. When they got around to analyzing the voxel (think: 3-D [...]

The poster misses an important problem. An overly conservative attempt to generate corrected results when measuring emotional responses in chocolates failed to yield any significant results whatsoever.

The resulting in-fighting destroyed a once productive research team. This has lead to a marked bias against Bonbon-Furore corrections.

[...] the salmon scanning would simply live on in Dartmouth lore as a “crowning achievement in terms of ridiculous objects to scan.” But the fish had a surprise in store. When they got around to analyzing the voxel (think: 3-D [...]

Ian Osmond - September 20th, 2009

I’d like to congratulate you on your clearly inevitable IgNobel.

DaveK - September 21st, 2009

Ahhh… this is what science is all about!

Marko Crivaro - September 21st, 2009

You call salmon ridicilous? Try a tree trunk. (I knew a guy who wanted to see if you could track the water diffusion in a tree with MRI, turned out you couldn’t). I have also inserted a good 10 kilo (20 pound or so) ham inside the scanner for.. reasons that are NDA. But still good job, for causing all this rucus. Now we all just need to hope all the not-so mathematical people working with MRI learn about this.

Josh Chang - September 21st, 2009

I was wondering first of all where I can get a preprint of the paper. I’m curious as to if the authors did any sort of analysis of significance levels. “Statistical significance” is a very arbitrary cutoff and I am wondering if the authors performed any sensitivity analysis of significance levels.

Thanks

Josh – I have added you to the list of folks to send the paper as soon as it is in press. We did not do any analysis of significance levels since our primary argument was related to the multiple comparisons problem. ~ Craig [Prefrontal]

Josh Chang - September 21st, 2009

Personally, I see this less as a weakness in multiple comparison p-value adjustment strategies, and more as a general weakness in Fisher’s formulation of frequentist statistics.

SCE - September 22nd, 2009

Clearly the next round of tests should be on members of congress!

Ohio Cyclist - September 22nd, 2009

And I suppose it was silly when Luigi Galvani applied an
electric current to a dead frogs leg, and it twitched.
Do we think that everything totally ceases when an
organism is no longer alive? Galvani proved otherwise.

Similarly though the fish is dead, this is another proof
that all activity has NOT ceased. Considering that fish
are cold-blooded, there may be an even higher chance of
functioning even longer after death, than might be expected
of warm-blooded organisms.

Ohio – Whether or not this is the case, the version of fMRI that we used for the salmon scans would not be sensitive to residual electrical currents in the brain. It is only sensitive to differences in magnetic susceptibility due to changes in the ratio of oxygenated and deoxygenated blood, hence the term blood oxygen level dependent (BOLD) imaging. There was no blood flow in our salmon, meaning that the signal could not rise and fall in synchrony with our experimental design. ~Craig [Prefrontal]

[...] a must in order to draw valid conclusions. Read Bennett’s description of event’s at his Blog and see the poster based on the data Here. « [...]

K - September 24th, 2009

While I do agree on the importance of correcting for multiple comparisons – in a random effects study, wouldn’t a big amount of the false positives even out across the subjects, thus somewhat lessening the impact that any single-salmon study could have on the field? Would be great to see a group result – otherwise the big neuroimaging journals would be justified to not publish that paper, and refer you to Neurocase instead… :)

Reminds me of the push-back over the paper by Vul et al. that you can find at (http://www.edvul.com/voodoocorr.php) — there will always be resistance to these kinds of corrections, and I think they always come in the peer review process because it’s fairly anonymous, and someone who uses questionable methods can defend them without being forced to make an argument in their defense.

Which would be hard, because so much of behavioral neuroscience is a recapitulation of just-so-stories from sociobiology, correlated by questionable methods and data.

Long story short: way to go. You’re not the ‘fish guys’, you’re the ‘good guys’.

Justin -I enjoyed your article, and made a more lengthy comment there. Briefly, I will just say that I do not know if the dorsal spinal region is close to the salmon substantia nigra. Unfortunately my years of human neuroanatomy do not transfer that well to the brain of a fish! ~ Craig [Prefrontal]

[...] parts of the brain of a dead salmon responds to human emotion. See also the Story Behind the Salmon here and the pdf of the study [...]

jim - September 28th, 2009

I saw something similar to this in the mid-1990’s. I think that it also showed that imaging found brain activity or spinal activity in a dead fish. I’m not 100% sure, nor am I sure where it was (I’m a molecular biologist, not an imager), but it was probably somewhere on the web – a site that someone alerted me to.

interesting, but could it be just a new exemple of cryptobiose (like tardigrada), so the salmon should be just a little alive (only the brain awaked, like a computer switch off but ready to start) !!

Freezing is knowing to conserve life, and salmons are not killed before freeze, are they ?

To live with him in the the future, just reed before the Umberto Eco novel : “how to travel with a salmon”, a funny story.

Sorry for my bad english

P. Jennings - October 8th, 2009

Salmon navigate, right? How do they do that? Sense of smell when that works – close to land. Or by magnetism, when out at sea.

There are magnetic particles in the lateral line:

>Magnetization measurements with a superconducting quantum >inference device magnetometer of various tissues of the >Atlantic salmon (Salmo salar L.) have shown the presence >of magnetic material associated with the lateral line. The >data suggest that the material is magnetite and of a size >suitable for magnetoreception. Magnetic particles were >isolated from the lateral line and nerve …
— Magnetic Particles in the Lateral Line of the Atlantic Salmon (Salmo salar L.) A. Moore, et al.

Have to wonder if there are also such particles in their brains? Enough to cause a change (even in a dead salmon) if exposed to fMRI?

There may very well be some magnetic particles in their brain. Also, I cannot discount that there may be some interaction of these particles with the scanner’s magnetic field. However, I do not believe that this effect would explain the repetitive rise and fall in the signal needed for the voxels to be considered significant. It is likely that voxels with such particles would have a relatively stable signal or a signal that slowly drifts over time – trends that are removed during image processing of the data. ~ Craig [Prefrontal]

[...] of the brain. There are many complex difficulties in interpreting fMRI scan data, as for example when researchers ran fMRI scans on a dead salmon fish, and discovered false positives. In 1985, Brian Dugan, who was already in prison for the murder of a 27-year-old woman and the [...]

[...] the story behind how this came about: http://prefrontal.org/blog/2009/09/the-story-behind-the-atlantic-salmon/ Share this:TwitterFacebookLike this:LikeBe the first to like this post. This entry was posted in [...]

[...] The voxel appears to be active (to a significantly high level) but where in reality, it is not. Bennett wanted to demonstrate these false positives by looking for active voxels on something that would [...]

[...] it is a well-known scientific truth that research results which are accompanied by a fancy, colorful fMRI scan, are perceived as more believable and more persuasive than simple bar graphs or text results (McCabe & Castel, 2007; Weisberg, Keil, Goodstein, Rawson, & Gray, 2008). Readers even agree more with fictitious and unsubstantiated claims, as long as you provide a colorful brain image, and it works even when the subject is a dead salmon. [...]