Monday, January 21, 2013

Amusing Titles in Psychological Science II: What is your PAT?

It may only be a
matter of time before the pun rises again says the BBC. Is that time
already upon us in psychological science? Let’s see.

In my previous post, I
provided a qualitative analysis of amusing titles in Psychological Science. Here is the quantitative part. I counted the
number of amusing titles per year in the journal in the past decade— least,
that’s what I said yesterday. But then I got busy and analyzed all volumes of Psych Science instead.

There were some issues I had to face. First, as Psych Science was trying to find its
form, it featured article types like general articles, feature reviews,
commentaries letters to the editor, and so on. I limited my analysis to
empirical articles, called research reports and research articles (although
there was a period in which they were called “original articles”).

Second, I am a cognitive psychologist and therefore less
familiar with social, clinical, and developmental psychology. This is somewhat
problematic because I wasn’t always sure whether something was a theoretical
construct or an allusion. For example, as a cognitive psychologist I know that
“flashbulb memory” is a construct. Otherwise, I might have thought it was part
of an amusing title. I probably wasn’t as discerning with regard to the other
fields, so my picks might reflect a certain “cognitive bias” and I might have
false-alarmed to some titles in the other sub-areas.

Third, there was an issue of what to do with retracted
papers by acknowledged fraudsters like Diederik Stapel and Lawrence Sanna. I
decided to keep them in. After all, the papers were accepted at the time. Also,
it doesn’t look like there’s a correlation between fraud and amusing titles. As
I said in my previous post, everyone uses amusing titles.

In discussing the results, I’m introducing a new index,
the PAT (Proportion of Amusing Titles). I computed the annual PATs for Psych Science from its inception in 1990
through 2012 (the last complete year).

Without further ado, here are the results. They clearly show
that amusing titles are on the rise in Psych
Science. There was an early start of .15 but then the PAT dropped down to
0, making 1993 the only year in Psych
Science history without amusing titles. The most amusing year in Psych Science history was 2012, with a
PAT of .41. This is the record to beat.

So Psych Science's PAT more than doubled over the years.Of course, the PAT corrects for the number of articles per volume. If we look at absolute numbers, we get an idea of how many amusing titles entered the scientific literature via Psych Science. The low was zero, as we already know. The absolute high was reached in 2010 with 105 amusing titles!

What are the PATs from specialty areas within psychological
science? Do social psychologists have higher PATs than cognitive psychologists?
And is the increase in PATs as seen in Psych
Science part of a more general trend? I addressed these questions by
comparing Psych Science to two other journals, which I selected because along with Psych Science, they are part of the reproducibility
project.

A group of researchers is trying to replicate the findings
published in the 2008 issues of Psych
Science, the Journal of Personality
and Social Psychology (JPSP) and the Journal
of Experimental Psychology: Learning, Memory, and Cognition (JEP:LMC). I computed
the PATs for the 2008 and 2012 volumes of all these two journals and then
compared them to Psych Science.

It is clear that—at least for the journals examined—social
psychologists have higher PATs than cognitive psychologists. Also
clear, however, is that the increase in PATs I found for Psych Science is not in evidence for the other
two journals (at least not for the periods I examined).

For a more complete picture, it might be interesting to have
PATs for other journals and maybe even individual researchers and entire areas
of research and fields. Then we can not only ask What is your H-index or What
is your impact factor but also What
is your PAT?