Efficiency and fairness in recurring data-driven risk assessments of violent recidivism
Marzieh Karimi-Haghighi, Carlos Castillo · 2021
In this paper, we consider the prediction of violent recidivism in criminal justice as currently done through machine learning methods. Specifically, we consider sequential evaluations performed on jail inmates with a state-of-the-art risk assessment instrument, RisCanvi. In this protocol, evaluations are done periodically every six months to all inmates. We study a scenario in which the inter-evaluation period depends on the characteristics of each inmate. In this scenario, only a fraction of the inmates, those with the highest probability of having changed risk, are selected for the next evaluation. Our work is based on a cost-benefit analysis which leads to fewer evaluations in exchange for some missed/undetected changes. When modeling risk change, we obtain prediction models with AUC in the order of 0.74-0.78, which can be used to schedule evaluations leading to a small number of missed changes (about 14%) by performing half of the evaluations (50%). This allows freeing resources and staff for other tasks. Importantly, we analyze if this method leads to discriminatory outcomes across some characteristics, including disparate impact in the evaluation rates along nationality and age. By adjusting decision boundaries we are able to mitigate the disparate impact and ensure equality in the rate of evaluation. Even after mitigation, missed changes remain small (about 15%) while still halving the number of evaluations needed.