Effects of Base Rate and Cutoff Point Choice on Commonly Used Measures of Association and Accuracy in Recidivism Research

• NCJ Number: 163192
• Journal: Journal of Quantitative Criminology Volume: 12 Issue: 1 Dated: (March 1996) Pages: 83-111
• Author(s): W R Smith
• Date Published: 1996
• Length: 29 pages
• Grant Number(s): 85-IJ-CX-0005
• Sponsoring Agency: National Institute of Justice (NIJ); US Dept of Justice NIJ Pub
• Publication Format: Article
• Publication Type: Test/Measurement
• Language: English
• Country: United States of America

Annotation

This study examined several measures of association and accuracy commonly used in the recidivism literature for their sensitivity to variations in the proportion observed to recidivate (the base rate) and in the proportion selected for classification as recidivists (selection ratio).

Abstract

Logistic regression models were used on a sample of 11,749 convicted offenders to generate predicted probabilities of four recidivism criteria that vary in base rates from 0.06 to 0.48. Cutoff point selections from 0.1 to 0.9 show the effects of cutoff point changes on the following commonly used measures of association and accuracy: RIOC (relative improvement over chance), MCR (mean cost rating), phi, gamma, PRE (proportion reduction in error), and percentage correct. Although all these statistics vary across base rate and cutoff points, some vary more than others; RIOC varies across cutoff points more than MCR, MCR more than phi, and phi more than gamma. Researchers who compare such statistics across studies should be wary of the dangers of ignoring such variation. 8 figures and 28 references