Tag Archives: sensitivity

Here's a pair of figures from a 2003 report by the National Academies 'Committee to Review the Scientific Evidence on the Polygraph' (full text), which includes several well-known statisticians.

The figure below shows the sensitivity versus false-positive rate for 52 controlled laboratory studies of naive examinees, untrained in polygraph countermeasures. Each study examinee was assigned to be truthful or deceptive, and a polygraph was used to test for deception. Thus, in this context, sensitivity is the likelihood of detecting deception in examinees who were, in fact, attempting to deceive the examiner.

Caption from the report: FIGURE 5-1 Sensitivity and false positive rates in 52 laboratory datasets on polygraph validity. NOTES: Points connected by lines come from the same dataset. The two curves are symmetrical receiver operating characteristic (ROC) curves with accuracy index (A) values of 0.81 and 0.91.

On the basis of this figure, the report concludes that "features of polygraph charts and the judgments made from them are correlated with deception in a variety of controlled situations involving naïve examinees untrained in countermeasures: for such examinees and test contexts, the polygraph has an accuracy greater than chance," and that "errors are not infrequent in polygraph testing." Indeed, the false positive rate is quite large for most studies.

The next figure is similar to that above, but for "field studies", which "involved examination of polygraph charts from law enforcement agencies’ or polygraph examiners’ case files in relation to the truth as determined by relatively reliable but nevertheless imperfect criteria, including confession by the subject or another party or apparently definitive evidence."

Caption from the report: FIGURE 5-3 Sensitivity and false positive rate in seven field datasets on polygraph validity. NOTE: Points connected by lines come from the same dataset.

Interestingly, these studies appear to confirm the laboratory experiments. However, note that the false-positive rate was greater than 10% in all studies, and as large as 70%.

Although these data should disqualify the polygraph for any high-stakes purpose (at least 1 in 10 truthful examinees would be incorrectly identified as deceptive), I'm surprised that there is so much empirical evidence of greater-than-chance lie detection. Nevertheless, the report goes on to criticize the quality and generalizability of polygraph research, and leaves the reader with a decidedly skeptical sense of the utility of polygraph in lie detection.