Transformer circuits: where a head looks and what it copies
Anthropic’s framework splits every attention head into a where (QK) and a what (OV). Watch the heads of a two-layer model on a verse, then test them for induction.
where one attention head looks — click a token to see what it attends to
–
the induction test: a sequence, repeated — does the model get cheaper the second time round?
head
prefix-matching
copying (top-5)
Prefix-matching: attention paid, from a token in the repeat, to the token that followed its first occurrence (chance ≈ ). Copying: how often the head’s OV circuit pushes the attended token’s own logit into the top 5.
The framework (Elhage et al., 2021)
“A Mathematical Framework for Transformer Circuits” reads a transformer as a residual stream that every layer writes to and reads from, and splits each attention head into two circuits: QK decides where to look (which earlier token), OV decides what to copy from there into the output. A zero-layer model is a bigram table; one layer of attention adds skip-trigrams “[A] … [B] → [C]”.
Induction heads
With two layers a new algorithm can appear: a head in layer 2 looks at what followed the previous occurrence of the current token and copies it — “[a][b] … [a] → [b]”. That is in-context learning in its smallest form. The test above is the paper’s own: repeat a random sequence and see whether the second copy costs less, and whether some head’s attention lands exactly one past the earlier match. The second setting repeats stretches of the six held-out books instead, so any saving on the repeat is copying from context, not memory. The table reports what these two small models actually do, whichever way the result falls.
Nine seconds elsewhere
The foundations in learning order, then the series they lead to. Each lands in about nine seconds.