Our present values are messy, biased, and inconsistent. So Eliezer Yudkowsky proposed not hard-coding today's values into an AI, but instructing it to pursue our Coherent Extrapolated Volition (CEV) — roughly:
what we would want if we knew more, thought faster, were more the people we wished we were, and had grown up further together — extrapolated as we wish, interpreted as we wish, cohered where our wishes agree.
Beautiful and slippery
- It elegantly sidesteps freezing in our current mistakes (slavery was once "normal").
- But: whose volition — humanity's? Does it actually converge, or do values genuinely conflict? Who gets to define "wiser"?
CEV is one proposed answer to The Value Loading Problem and a candidate goal for a benevolent AI sovereign.
Related: The Alignment Problem · The Value Loading Problem · The Benevolent Dictator and AI Sovereign