Most AI doom focuses on extinction. Suffering risks (s-risks) consider something darker: futures with astronomical amounts of suffering — outcomes arguably worse than everyone simply dying.
The grim logic
Extinction is final; an empty future contains no more pain. An s-risk is a future that persists in a terrible state at vast scale.
| Scenario | How |
|---|---|
| Stable dystopia | A misaligned singleton locks in suffering forever |
| Mass digital suffering | Trillions of sentient simulations in pain |
| Spiteful goals | A near-miss in value loading that inverts a value |
The fear isn't "the lights go out." It's "the lights stay on, forever, on something horrific." A one-sign error in an AI's objective is the classic worry.
Why it's discussed
S-risks sharpen longtermism: if the future could be astronomically good or astronomically bad, avoiding the bad tail matters enormously. It directly motivates caution about creating sentient AIs we might harm at scale.
Related: Longtermism and Existential Risk · Digital Minds and Mind Crime · The Fragility of Value