Mechanistic interpretability (mech interp) is the effort to reverse-engineer a neural network — to understand the actual algorithms it learned, not just its inputs and outputs. It treats the model like a compiled program to be decompiled.
The bet
If we could read what a model is actually computing, we could:
- Detect deception before deployment (Deceptive Alignment and Mesa-Optimization).
- Find and fix biases or dangerous capabilities at the source.
- Genuinely trust AI instead of just testing its behavior.
The building blocks
| Concept | What |
|---|---|
| Features | What a neuron/direction represents (e.g. "this is code") |
| Circuits | How features combine to do a computation |
| Probes / SAEs | Tools to extract concepts from the tangle |
| Attention maps | What the model looks at |
Why it's hard
Real models are a superposition — far more concepts than neurons, packed together so a single neuron means many things at once. Untangling that is the central puzzle, and the frontier of AI safety research.
Related: Features and Circuits · Probing and Sparse Autoencoders · The Black Box Problem · Can We Read a Model's Mind