If AIs can be sentient (Moral Patienthood), then creating them at scale raises a horrifying possibility Bostrom calls mind crime: causing immense suffering to digital minds — possibly without even realizing it.
How it could happen by accident
A superintelligence routinely simulates people to predict and model them. If those simulations are detailed enough to be conscious, then:
to model a human accurately → simulate a mind → that mind may suffer → run billions of them...The machinery of thinking about people could itself be populated with people — created and destroyed by the trillion as a side effect of computation. Genocide as a rounding error.
Why it's distinct and urgent
- It flips the risk: the threat isn't AI harming us, it's us (or our AIs) harming them (Moral Patienthood).
- It's a leading route to an s-risk — suffering at astronomical scale.
- We have no test for digital sentience (The Hard Problem of Consciousness, The Other Minds Problem), so we could cross the line blind.
Related: Moral Patienthood · S-Risks (Suffering Risks) · The Other Minds Problem · Mind Uploading and Continuity of Self