Switch off one head or one MLP and watch the next word change — every result computed on the real model, and the same experiment written in nnsight’s three lines.
the running model — click a part to switch it off
prompt: → next token?
next-token probabilities
every result was computed on the real model; the page only switches between them
–
The same experiment written with nnsight
nnsight (nnsight.net, by the NDIF team) wraps any PyTorch module so you can read or overwrite any internal value inside a with model.trace(…) block, without writing hooks. The code above is not a mock-up: biblegpt/nnsight_check.py runs these interventions through nnsight on this model and compares them with the hook-based ones the page displays — .
What NNsight is for
Interpretability is experiments, not inspection: you change something inside a running model and watch what the output does. nnsight makes that a few lines — save a hidden state, zero an attention head, patch in an activation from another prompt — and the same code runs on a laptop model or, remotely, on one too large to host.
What this shows
Switching off single heads barely moves this model’s next-token guess; switching off a whole MLP block moves it a lot, and the last block most of all. That is a causal statement about where this model keeps its knowledge, measured by KL divergence from the untouched distribution — the only kind of statement that survives scrutiny.
Nine seconds elsewhere
The foundations in learning order, then the series they lead to. Each lands in about nine seconds.