www.elsblog.org - Bringing Data and Methods to Our Legal Madness

20 June 2012

External Validity: Comparing Psychology Laboratory and Field Studies

A hugely important topic for empirical psychology is the external validity of laboratory-based studies. "A widely-held assumption within the social sciences is that the rigor of experimental research is purchased at the price of generalizability of results," and that "this trade-off plays out most directly in those fields that use laboratory experiments to study how humans navigate complex social environments." In Revisiting Truth or Triviality: The External Validity of Research in the Psychological Laboratory, Gregory Mitchell (UVa) revisits an earlier paper (Anderson et al., 1999), compares results from lab and field studies, and finds, in part, that the external validity of psych lab studies "differed considerably" among various psychological subfields. The abstract follows.

"Anderson, Lindsay, and Bushman (1999) compared effect sizes from laboratory and field studies of 38 research topics compiled in 21 meta-analyses and concluded that psychological laboratories produced externally valid results. A replication and extension of Anderson et al. (1999) using 217 lab-field comparisons from 82 meta-analyses found that the external validity of laboratory research differed considerably by psychological subfield, research topic, and effect size. Laboratory results from industrial–organizational psychology most reliably predicted field results, effects found in social psychology laboratories most frequently changed signs in the field (from positive to negative or vice versa), and large laboratory effects were more reliably replicated in the field than medium and small laboratory effects."