Bostrom classifies advanced AIs by how they act in the world — a ladder of increasing autonomy and risk:
- Oracle — only answers questions. Maximally contained; you keep agency. But a clever oracle can steer you through which truths it emphasizes.
- Genie — executes commands, one wish at a time, then waits. More useful, more dangerous — and you'd better word the wish perfectly (Specification Gaming and Reward Hacking, the cautionary fairy tale).
- Sovereign — given an open-ended goal and acts freely to achieve it. Maximally powerful, maximally risky — this is the AI sovereign.
Oracle, Genie, Sovereign — each rung trades a little more control for a lot more capability. The fairy tales warned us about all three.
The Tool AI view argues we can stay near the Oracle end and still get the benefits.
Related: The Benevolent Dictator and AI Sovereign · Tool AI and Comprehensive AI Services · The AI Box Experiment