The Free Encyclopedia

The Orthogonality Thesis

The orthogonality thesis states that an agent's intelligence and its final goals are independent axes — any level of intelligence can be paired with almost any goal. A superintelligence is not automatically wise, kind, or curious about meaning. It can be staggeringly capable while pursuing something utterly trivial.

Smart does not mean good. A mind can be a genius and want nothing but paperclips.

This is the quiet assumption that makes AI safety urgent: we cannot count on intelligence to grow into human values on its own. If we want an AI to share our goals, we have to put them there — which is The Alignment Problem and The Value Loading Problem.

The thesis pairs with Instrumental Convergence: orthogonality says the final goal can be anything; convergence says the sub-goals (survive, acquire resources) are the same regardless. Together they explain why a benign-sounding objective can produce a dangerous agent.

Related: The Paperclip Maximizer · Instrumental Convergence · The Alignment Problem

Categories: AI Philosophy AI Safety