This report synthesizes the winning reports submitted to the National Institute of Justice (NIJ) Forecasting Challenge.
This paper aims to add to the knowledge of risk assessment creation by synthesizing the 25 winning, nonstudent papers submitted to the National Institute of Justice 2021 Recidivism Forecasting Challenge (hereafter referred to as the Challenge), which evaluated team forecasts of the probability of individuals on parole recidivating during a specified time interval. This review of winner’s reports suggests a need to continue evaluating challenging elements of machine learning models. The summary of these winning papers provides important information related to the creation of risk assessments. These include considerations of the types of models used and whether there is a superior model. Winning reports also brought up points related to variables created and used and that, ultimately, were important in their models. NIJ anticipated that the Challenge would inspire teams to apply innovative data science techniques to forecast the probability that individuals would recidivate and inherently further general knowledge of what variables are important in forecasting recidivism. The summary of winners’ reports suggests that as research on risk assessment continues, there should be an increased focus on design considerations. Additionally, this Challenge focused on fairness and accuracy through narrow definitions, where several other options exist. The authors of the synthesis argue that the Challenge met its aim of a better understanding of factors contributing to recidivism. In addition, researchers also gained a better understanding of the limitations of the methods, models, and feature creation used to forecast recidivism.