This article discusses risk assessment instruments and cohort bias in the context of efforts to analyze criminal histories in a cohort-sequential longitudinal study of children; and demonstrates that tools trained to predict the likelihood of arrest between the ages of 17 and 24 y on older birth cohorts systematically over-predicts the likelihood of arrest for younger birth cohorts over the period 1995 to 2020.
Risk assessment instruments (RAIs) are widely used to aid high-stakes decision-making in criminal justice settings and other areas such as health care and child welfare. These tools, whether using machine learning or simpler algorithms, typically assume a time-invariant relationship between predictors and outcome. Because societies are themselves changing and not just individuals, this assumption may be violated in many behavioral settings, generating what the authors call cohort bias. Analyzing criminal histories in a cohort-sequential longitudinal study of children, the authors demonstrate that regardless of model type or predictor sets, a tool trained to predict the likelihood of arrest between the ages of 17 and 24 y on older birth cohorts systematically over -predicts the likelihood of arrest for younger birth cohorts over the period 1995 to 2020. Cohort bias is found for both relative and absolute risks, and it persists for all racial groups and within groups at highest risk for arrest. The results imply that cohort bias is an underappreciated mechanism generating inequality in contacts with the criminal legal system that is distinct from racial bias. Cohort bias is a challenge not only for predictive instruments with respect to crime and justice, but also for RAIs more broadly. (Published Abstract Provided)