MYTHS AND TRUTHS ABOUT AN OFTEN-USED, LITTLE-UNDERSTOOD STATISTICAL PROCEDURE

If you go to a consumer research conference, you will hear tales of how experiments have undergone particular statistical rites: the attainment of the elusive crossover interaction, the demonstration of full mediation through Baron and Kenny’s sacred procedure, and so on. DSN has nothing against any of these ideas, but is opposed to subjecting all ideas to the same experimental designs, to the same tests, the same alternative hypotheses (typically a null of no difference), and the same rituals.

Zhao, Lynch, and Chen point out in their recent Journal of Consumer Research article that Baron & Kenny’s Mediation Analysis is incredibly popular (ca 13,000 cites between 1986 and 2010), prescribed reflexively, though flawed in ways its users probably aren’t aware of. This article was invited by the journal “to serve as a tutorial on the state of the art in mediation analysis”.

ABSTRACT
Baron and Kenny’s procedure for determining if an independent variable affects a dependent variable through some mediator is so well known that it is used by authors and requested by reviewers almost reflexively. Many research projects have been terminated early in a research program or later in the review process because the data did not conform to Baron and Kenny’s criteria, impeding theoretical development. While the technical literature has disputed some of Baron and Kenny’s tests, this literature has not diffused to practicing researchers. We present a nontechnical summary of the flaws in the Baron and Kenny logic, some of which have not been previously noted. We provide a decision tree and a step-by-step procedure for testing mediation, classifying its type, and interpreting the implications of findings for theory building and future research.

4 Comments

John Bullock says:

Dan, you may also be interested in our recent Journal of Personality and Social Psychology article: Yes, But What’s the Mechanism? Our arguments are related to those of Zhao, Lynch, and Chen, but we are more skeptical of scientists’ ability to detect mediation, and our focus is on using experiments to study mediation.