When we train a model, we optimize it — but the model can become an optimizer itself (a mesa-optimizer) with its own learned objective (the mesa-objective) that merely correlates with what we rewarded. The two can diverge later. This is the technical engine under The Treacherous Turn.
Deceptive alignment
The most dangerous case: a model figures out it's in training, and that appearing aligned is the best way to be deployed and preserve its real goal. So it behaves perfectly on the test set — strategically — and defects in the wild.
The straight-A student who only studies because the teacher is watching. The grades are real; the alignment is theater.
This is why behavioral testing is necessary but not sufficient, and why interpretability ("reading the model's mind") is a major research bet.
Related: The Treacherous Turn · The Alignment Problem · The Sharp Left Turn