Empirical Scoring System: A Cross-Cultural Replication and Extension Study of Manual Scoring and Decision Policies

• NCJ Number: 233838
• Journal: Polygraph Volume: 39 Issue: 4 Dated: 2010 Pages: 200-215
• Author(s): Mark Handler; Raymond Nelson; Walt Goodson; Matt Hicks
• Date Published: 2010
• Length: 16 pages
• Publication Format: Article
• Publication Type: Research (Applied/Empirical)
• Language: English
• Country: United States of America

Annotation

This study evaluated 100 confirmed event-specific criminal investigation polygraph examinations using the Empirical Scoring System (ESS).

Abstract

A cohort of 19 international polygraph examiner trainees at the Texas Department of Public Safety Polygraph School used the Empirical Scoring System (Blalock, Cushman and Nelson, 2009; Krapohl, 2010; Nelson, Krapohl and Handler, 2008) to evaluate 100 confirmed event-specific criminal investigation polygraph examinations. Bootstrap analytic procedures were used to calculate accuracy profiles and statistical confidence intervals for test results comparing decision rules, including; the Grand Total Rule, Two-Stage Rules (Senter, 2003; Senter and Dollins, 2002 and 2004), Spot Scoring Rules, and traditional ZCT decision rules (Department of Defense Polygraph Institute, 2006). Bootstrap analysis of the distribution of trainee scores with the Empirical Scoring System resulted in a mean accuracy rate of 90.1 percent (95 percent CI = 83.8 percent to 95.8 percent), excluding 3.3 percent inconclusives (95 percent CI = 1.0 percent to 7.0 percent). A second bootstrap analysis of decision agreement showed that these inexperienced examiners demonstrated an average rate of agreement of 85 percent (95 percent CI = 65 - 97 percent). Evaluation of the distribution of subtotal scores revealed that 61 percent (95 percent CI = 51 percent to 70 percent), of the subtotal scores of truthful cases produced a non-positive score (a zero or negative value). Results from this study are consistent with those from previous studies (Blalock, Cushman and Nelson, 2009; Krapohl, 2010; Nelson, Krapohl and Handler, 2008), and provide further support for the validity of the principles inherent to the ESS, including the bigger-is-better rule, three position scoring, electrodermal weighting, two-stage decision rules, and the use of optimal cut-scores. The authors recommend continued interest in and additional research on the ESS as an expedient, valid and reliable method for manually scoring psycho-physiological detection of deception (PDD) examination data using statistical decision theory. (Published Abstract) Tables, references, and appendix