Thursday, 18 February 2016

Desperate marketing from Journal of Neuroscience

Yesterday
Chris Chambers was outraged to receive marketing spam from Journal of
Neuroscience, boasting that articles published in their journal received many more citations than those published in their competitors.

There are many reasons for taking a dim view of using citation
counts as a measure of journal prestige. Citations are very dependent on the
field of study, and their distribution is highly skewed. It could be argued
that Journal of Neuroscience was avoiding these problems: it compared itself with
other journals that covered similar subject areas, and it presented total counts,
rather than means.

Except……It did not
make any adjustment for the fact that Journal of Neuroscience publishes many
more papers than the other journals it compares itself to. I looked at Scopus
statistics for articles and reviews published in four journals for the period
2010 to 2013. Journal of Neuroscience published 7004 papers, Neuroimage
published 4258, Neuron published 1348 and Nature Neuroscience published 962. So
saying that Journal of Neuroscience had more citations is a bit like claiming
that India is a wealthier country than Luxembourg.

I started to wonder how Journal of Neuroscience compared
with these other journals when number of papers was taken into account. The answer
is shown below:

So the question we are left with is whether those promoting
the Journal of Neuroscience are being devious, and think their readers are too stupid
to notice this very basic error, or whether they themselves are too stupid to notice
it.

P.S. 20 Feb 2016

I’m adding here an analysis I didn’t have time to do
earlier, which simply looks at what the probability is that a paper published
in each of these four journals gets cited a given number of times. The results
are very similar to those obtained when looking at means: Journal of
Neuroscience and Neuroimage are at level pegging, whereas Nature Neuroscience
and Neuron are associated with much higher citation rates.

So does this mean, as implied by the Journal of Neuroscience
marketing, that authors should preferentially send their papers to Neuron or
Nature Neuroscience, because this will lead to them being more highly cited?
The answer is almost certainly no. I took at quick look at the most highly
cited papers in these four journals, and some common themes occurred. If you
want to be highly cited, you should write a paper that is either (a) a review
of the field or (b) reports a methodological advance. And the reviews, which I
suspect are mostly commissioned, are by famous people. Furthermore, my impression
was that empirical papers on molecular neuroscience were more highly cited than
those on cognitive neuroscience – I leave that for someone even more nerdy than
me to confirm.

My advice to any academic who wants people to notice their
work is to do sound, reproducible science on an important topic and publish it
promptly, rather than wasting months or years trying to get it into high impact
journals. The prestige of the journal you publish in may impress some people,
but it won’t magically transform a poor study into a good one. And if your work
is good enough, people who care about the subject matter will read, and cite,
it, regardless of where it is published.