In 2016, ProPublica published an investigation that changed how we think about algorithmic justice. The COMPAS system, used by courts across America to predict recidivism and inform sentencing decisions, was wrong about future criminals 61% of the time. But it wasn't wrong equally: it falsely labeled Black defendants as future criminals at twice the rate it mislabeled white defendants. The algorithm wasn't just predicting crime—it was encoding bias into the justice system at scale.

This is the trolley problem in its most insidious form: not a split-second decision about who to save, but a systematic choice about whose freedom to sacrifice in the name of public safety. And unlike the trolley, we can't see the tracks diverging.

The Prediction Problem

Predictive policing comes in two forms: algorithms that predict where crimes will occur (hotspot policing) and algorithms that predict who will commit them (risk assessment). Both promise to make policing more efficient, directing limited resources where they're needed most. Both create trolley problems we're barely beginning to understand.

COMPAS—Correctional Offender Management Profiling for Alternative Sanctions—asks defendants 137 questions and produces a risk score from 1 to 10. Judges use these scores to make decisions about bail, sentencing, and parole. The algorithm considers factors like age, criminal history, and answers to questions about family, friends, and attitudes. It doesn't explicitly consider race. But it doesn't need to.

When your training data comes from a justice system with documented racial disparities, your algorithm learns those disparities as patterns. Arrest rates, conviction rates, incarceration rates—all higher for Black Americans, not because of higher crime rates but because of over-policing, prosecutorial discretion, and systemic bias. Feed that data to an algorithm, and it learns that being Black is correlated with recidivism. The algorithm becomes a bias launderer, transforming historical discrimination into seemingly objective predictions.

The Self-Fulfilling Prophecy

Chicago's Strategic Subject List—the "heat list"—identified individuals at high risk of being involved in gun violence, either as perpetrators or victims. Police used the list to conduct "custom notifications," visiting high-risk individuals to warn them they were being watched. The goal: prevent violence before it happens.

The problem: the list was generated from arrest data, social network analysis, and prior police contact. If you lived in an over-policed neighborhood, if your friends had been arrested, if police had stopped you before—even without charges—you were more likely to be on the list. And once you were on the list, police watched you more closely, increasing the likelihood of arrest, which validated the algorithm's prediction.

This is the trolley problem as feedback loop. The algorithm predicts you're high-risk, so police focus on you, so you're more likely to be arrested, so the algorithm's prediction appears correct. The tracks don't just diverge—they create themselves.

PredPol, used by police departments across the US, predicts where crimes will occur by analyzing historical crime data. It tells police where to patrol. More patrols mean more arrests, which feed back into the algorithm as new crime data, which predicts more crime in the same areas. The algorithm doesn't predict crime—it predicts where police will find crime when they look for it.

Accuracy vs. Fairness: The Impossible Trade-Off

Here's where the trolley problem becomes mathematically precise. ProPublica found that COMPAS was equally accurate for Black and white defendants—about 61% wrong for both groups. But it made different kinds of errors: Black defendants were more likely to be falsely labeled high-risk (false positives), while white defendants were more likely to be falsely labeled low-risk (false negatives).

Northpointe, COMPAS's creator, argued the algorithm was fair because it had equal predictive value across races—if it said you were high-risk, you were equally likely to reoffend regardless of race. ProPublica argued it was unfair because it had different error rates—Black defendants faced a higher risk of being wrongly labeled dangerous.

Both were right. And that's the problem.

It's mathematically impossible to satisfy both definitions of fairness simultaneously when base rates differ between groups. You can have equal false positive rates or equal predictive value, but not both. The trolley problem here isn't about choosing between two bad outcomes—it's about choosing which definition of fairness to encode, knowing that choice will systematically harm one group more than another.

Who Bears the Cost?

The UK's Gangs Matrix, used by London's Metropolitan Police, identified individuals as gang members based on social media activity, music videos, and association with known gang members. Being on the matrix meant increased police attention, restrictions on movement, and barriers to employment and housing. The matrix was 78% Black, in a city where Black people are 13% of the population.

Young Black men were added to the matrix for appearing in music videos, for social media posts, for being Facebook friends with people already on the list. The algorithm treated cultural expression as evidence of criminality, association as guilt. And once you were on the matrix, getting off was nearly impossible—even if you were never charged with a crime.

This is the trolley problem where one track is invisible. The people on it don't know they're there until the consequences arrive: denied jobs, stopped by police, restricted from certain areas. The algorithm makes its predictions in secret, and the people it predicts about have no right to see the data, challenge the score, or understand why they're being targeted.

The Minority Report Question

Philip K. Dick's "Minority Report" asked whether it's ethical to punish people for crimes they haven't committed yet. Predictive policing asks the same question, but frames it as resource allocation: we're not punishing predicted criminals, just watching them more closely, directing police resources more efficiently.

But increased surveillance is a form of punishment. Being on a watchlist restricts your freedom, limits your opportunities, marks you as suspicious. And when that surveillance leads to arrest—for crimes that might not have been detected without the extra attention—the prediction becomes self-fulfilling. You weren't arrested because you were going to commit a crime. You were arrested because the algorithm said you would, so police were watching when you did.

The trolley problem here is temporal: do we sacrifice the freedom of people who might commit crimes to prevent harm to potential victims? Do we accept that some people will be wrongly targeted to catch others who are correctly identified? And who decides what probability threshold justifies intervention—50%? 30%? 10%?

The Tracks We Can't See

Unlike the original trolley problem, predictive policing doesn't present us with a visible choice. The algorithm runs in the background, generating scores, creating lists, directing resources. Most people don't know these systems exist. Those who are targeted often don't know why. The tracks diverge invisibly, and by the time you realize you're on one, the trolley has already passed.

The defenders of these systems argue they're better than human judgment—less biased, more consistent, more efficient. And they're right that human judges and police officers are biased. But algorithmic bias is different: it's systematic, scalable, and invisible. A biased judge can be challenged, appealed, removed. A biased algorithm is protected as a trade secret, defended as objective, and deployed across entire jurisdictions.

The trolley problem of predictive policing isn't about choosing between public safety and civil liberties. It's about choosing who gets to make that choice, whose definition of fairness gets encoded, and whether we'll even admit we're making a choice at all. The algorithm is already pulling the lever. The question is whether we'll keep pretending it's not.