By Bernat Ivancsics
In an August exposé, FiveThirtyEight and The Marshall Project investigated a proposed risk assessment tool in Pennsylvania’s criminal sentencing system that would aid courts and judges in their work. In effect, statistical analysis would be used to measure the likelihood of a criminal committing another crime in the future, and the output of the analysis—a point-based tier-system—could determine the number of years in prison or the cost of bail for the criminal.
In the followings, I’ll address the way 538 covered the issue and analyzed the statistical framework of the new tool. The genre of this post, therefore, is data journalism criticism, not a response to the changes in PA’s criminal justice system.
Risk assessment is probability and probability is statistics. As 538 rightly points out, the state of PA would rely on the probability of possible future crimes while calibrating its assessment of probability based on the data of crimes committed in the past.
According to the article, advocates of risk assessment value the potential employment of the tool—essentially, a set of surveys and their evaluations—in courts with long backlogs of cases and barely any time to properly judge individual cases. With this pace and level of attention, backing judges’ decision with a new layer of scientific reasoning sounds reasonable.
Critics of the system, however, argue that calibrating the tier-system based on data rendered from past cases would incorporate the local legal system’s biases and socio-economical contingencies. Furthermore, the proposed tier-system is not granulated enough to account for nuances, that is, individual instances of the same crime committed in different situations and by different persons.
The article employs a basic journalistic tool to ground this abstract concept into human experience: it brings Mr Fosque as an example, a veteran arrested previously for DUI, who would immediately accumulate 6 points in the survey for being male, for having a certain age, and for having a history of minor traffic violations in the past.
538 even produced an interactive infographic to test what the effects would be of setting the tier-system too loosely or to strictly: based on the ranges of “high risk offender” and “low risk offender” on a slide, readers can calculate the chances of ending up with low risk criminals behind the bars or, conversely, too many high risk criminals committing new crimes while on parole.
The article is fair and balanced in showcasing the pros and cons of risk assessment in judging criminals. It does not, however, reflect on the problem of the tier system’s granularity (cf. Fosque’s results) or the oddities/dirtiness of empirical data on previous crimes and criminals.
In theory, risk assessment could streamline the sentencing process, but at the same time it can depersonalize criminal sentencing to the extent that “statistical errors” could result in serious mistrials or errors in judgements. In these cases, where would “statistical/algorithmic accountability” stem from?