Algorithmic bias is when an AI system produces systematically unfair outcomes for certain groups. The model isn't malicious — it's a mirror, reflecting (and often amplifying) patterns in its training data.
Where it comes from
- Biased data — train on historical hiring decisions and you learn historical discrimination.
- Proxy variables — drop "race" and the model rediscovers it via zip code, name, or shopping habits.
- Skewed objectives — optimizing one metric (Goodhart's Law) can quietly trade away fairness.
Real cases
| Domain | Bias |
|---|---|
| Hiring | Résumé screeners penalizing women's colleges |
| Justice | Recidivism scores skewed by race |
| Lending | Higher rejection for equally-qualified minorities |
| Vision | Face systems failing on darker skin (Facial Recognition and Surveillance) |
"The algorithm decided" can launder human bias into the appearance of objectivity — which makes it harder to challenge, not easier.
What helps
- Diverse, representative data and bias audits.
- Fairness metrics — though "fair" has multiple, mathematically incompatible definitions.
- Transparency (The Black Box Problem) and human appeal paths.
Related: Reinforcement Learning from Human Feedback (RLHF) · The Value Loading Problem · The Black Box Problem