Attention Visualization

One of the most accessible interpretability windows: visualizing attention — drawing the weights that show which tokens the model "looked at" when processing each word.

#What you see

For each token, attention weights reveal what it drew context from:

"The cat sat because it was tired."
                       ↑
        "it" attends strongly to → "cat"

Tools like BertViz render these as heatmaps or connection lines across the attention heads, and different heads specialize — one tracks grammar, another long-range references, another quotation matching.

#Useful, but not the whole story

Attention maps show where the model looked, not what it concluded. They're a flashlight into one part of the machine — real understanding needs circuits too.

Related: The Attention Mechanism · Mechanistic Interpretability · How Transformers Work