Decision Science News was recently at a cocktail party and mentioned that the percentage of studies in a top social psychology journal that employed deception was 80%.

Coarsely, deception in psychological research usually means telling falsehoods (* some say “lies”) to experimental participants. It is supposed to be followed up by debriefing, which is the technical term for confessing. Apologizing is not part of the protocol. Experimental economists tend never to use deception, social psychologists use it some of the time. But how much?

At the cocktail party, our interlocutors accused us of deceiving them about the 80% number. Something to the tune of “you just made that up!”

“Did we just make that up?”, we wondered.

Utilizing a new technology called “the internet”, DSN found the 80% figure cited in two articles Ortmann & Hertwig, 1997 and Taylor and Shepperd, 1996. We didn’t just make it up! However, these articles didn’t do the counting themselves, they cited another article. For example, Taylor and Shepperd say “upwards of 81% of studies published in the top psychological journals use deception (Adair, Dushenko & Lindsay, 1985).”

But what did Adair, Dushenk & Lindsay say?

Leveraging a sophisticated multi-stage technique known as “downloading the article” we found that these authors went through all articles published in JPSP, JESP, and PSPB in 1983, extracted each study, and coded whether it used deception or not. (JPSP = Journal of Personality and Social Psychology; JESP = Journal of Experimental Social Psychology; PSPB = Personality and Social Psychology Bulletin). The results are summarized in their Table 5 on page 86.

The result: a stunning 81% of studies in JESP that year used deception. 50% of JPSP studies and 54% of PSPB studies used the practice. Aggregating, 55% of studies in the three journals used the practice in 1983, which is a bit lower than 1979 (58%).

1979, by the way, seems to have been a tough year for a few experimental social psych participants. The Adair et al article (p. 65) mentions a 1979 study by Marshall & Zimbardo in which

Subjects were (a) misled about the purpose of the study, (b) told that they would receive a vitamin injection when, in fact, they received an injection of epinephrine, (c) misinformed regarding the somatic effects of the drug, and (d) misled through fake equipment to believe that their physiological responses were no longer measured when in fact they were. In addition, (e) a doctor also faked the administration of a comparable injection to (f) a confederate, who then (g) proceeded to act in a bizarrely “euphoric” fashion to create a context for subjects’ perceptions of the drug’s effects. In a footnote the authors noted that (h) the medical school’s ethics committee did not allow them to introduce an anger manipulation as well. On top of all this, a complete debriefing of subjects was postponed for several weeks until all of the subjects had been tested.

They also mention (p. 64) the following study, first noted by Kelman (1967):

An experiment (Campbell, Sanderson, & Laverty, 1964) designed to study the establishment of a conditioned response in a situation that was traumatic but not painful and in which stress was induced through the use of a drug that produced a temporary interruption of respiration. This experience, although not painful, was regarded as “horrific” by subjects. Subjects were not warned in advance about the effect of the drug, because this information would have reduced the traumatic impact of the experience.

It would be good to know what the rate of deceptive experiments is today. If anyone knows, please share.

(*) Sorry if the term “telling falsehoods” (or “lying” as some say) offends the reader’s sensibilities, however, it can be occasionally useful to suspend a euphemism for a moment to remind oneself of the underlying logical equivalence. One might object that deception is softer than lying. According to m-w.com to deceive is “to give a false impression” while to lie is “to make an untrue statement with the intent to deceive”. By these definitions, one could imagine deception in experiments that does not involve lying. For instance, since cosmetics can cause people to look younger than they are, asking participants to judge the age of a model who is wearing cosmetics is mild deception, but not lying. Telling the participant that the model is not wearing cosmetics is lying. This latter case is what is called “deception” in psychology, with few exceptions.

UPDATE
After posting this, I got an email from a family friend who happens to be a world expert on the topic, and adds some valuable references. The Nicks, Korn and Mainieri article reports the following percentages of deceptive studies for 1994: JPSP 31% and JESP 50%. As in 1983, JESP is still quite a bit higher.

* Korn, J. H. Illusions of reality: A history of deception in social psychology. SUNY Press, 1997. especially chapter 2 on the grownth of deception from 1921 to 1989.

Your internet search was somewhat incomplete. Hertwig and Ortmann wrote many more
interesting articles on deception ;-), and they did so (much) more recently; in some of these
articles they actually did some counting (e.g., those marked with “*” which are attached).
Here is a more complete list of their articles on deception; feel free to distribute widely:

Hertwig & Ortmann: “Deception in Social Psychological Experiments: Two Misconceptions and
a Research Agenda,” Social Psychology Quarterly, 71.3., 2008, 222 – 227.

*Hertwig & Ortmann, “Deception in Experiments: Revisiting the Arguments in Its Defense,”
Ethics and Behavior 18.1., 2008, 59 – 92. (This being a companion piece to the next.)

4 Comments

[…] How much deception is there in social psychology? – A hell of a lot! Deception is necessary in many experiments in order to scientifically control for specific variables. This article shows the research behind the research. […]

I’m uncertain what the point of this article is. Lots of stuff implied, perhaps, but very little is stated directly. So I’ll just ask. Is it correct to infer that you believe all deception is immoral? that all attempts by researchers to mask from research participants the hypotheses they are testing qualify as deception and therefore are inherently immoral? that experimental economists are completely transparent about their hypotheses given that they “tend never” (which is it, btw?) to use deception?

I’m not sure what the answers are to the above questions but between the snarky tone, the sarcasm, the lack of an expressed goal, and the implicit assumptions, this column is comes pretty close to violating Grice’s maxim of manner. Of course that doesn’t make it deceptive. But I found it more inflammatory than informative.

The point is to give an approximate answer to the question of how often the practice of deception is employed in social psychology studies in top journals. Fans of the science of science, we find this interesting in its own right.

Lots of stuff implied, perhaps, but very little is stated directly. So I’ll just ask. Is it correct to infer that you believe all deception is immoral? that all attempts by researchers to mask from research participants the hypotheses they are testing qualify as deception and therefore are inherently immoral?

No. Concerning “all”, there are useful studies that employ deception, for example those that show discrimination in housing applications. Concerning “moral”, you may be interested to know that even if you find it morally ok to use deception, there are good scientific reasons not to. For example, there is the issue of “poisoning the pool” (as in subject pool). Why would anyone so deceived ever trust what an experimenter says in the future?

The only thing I’m implying is that observed percentages are surprisingly high, but then I do say “stunning”, so I’m not being all that indirect about it.

that experimental economists are completely transparent about their hypotheses given that they “tend never” (which is it, btw?) to use deception?

1) As decision scientists, we try not to say “never”. What is to stop a rogue experimental economist from using deception? But the experimental economics journals are strict on not publishing deceptive articles and there is a code among experimental economists. See the Ortmann and Hertwig articles for more on this. BTW, Being opaque about one’s hypotheses is not deception. For instance, observational studies (e.g. watching people form lines in cafeterias) is not deception even though nobody has any idea what your hypothesis is.

I’m not sure what the answers are to the above questions but between the snarky tone, the sarcasm, the lack of an expressed goal, and the implicit assumptions, this column is comes pretty close to violating Grice’s maxim of manner. Of course that doesn’t make it deceptive. But I found it more inflammatory than informative.