IOI Circuit Map

GPT-2 small indirect-object circuit (Wang et al.) · a documented reference model, not my internals · layer ranges from recollection, not source-certified
01 / Circuit graph (head groups and causal flow)
prev-tok feeds induction S duplicated inhibit attn to S copy IO name writes against if NM ablated prompt tokens Mary, John ... John gave Previous-Token heads ~L2 to L4 Duplicate-Token heads ~L0, L3 Induction heads ~L5 to L6 S-Inhibition heads ~L7 to L8 Name-Mover heads ~L9 to L10 Neg. Name-Mover heads ~L10 to L11 Backup Name-Mover heads ~L9 to L11 logit IO name
Read left to right as causal flow, validated by patching a node and measuring the logit shift. Detection is parallel, not a chain: previous-token, duplicate-token, and induction heads all establish that the subject name S is repeated (the previous-token heads feed the induction heads). The S-inhibition heads then modulate the query of the name-mover heads, steering their attention away from S so they copy the other name, the indirect object, to the output. Negative name-mover heads write against the answer (calibration), and backup name-mover heads take over if the primary ones are ablated. Layer ranges are from recollection of Wang et al. and are not source-certified; exact head indices are omitted for that reason.
02 / Feature card (illustrative, SAE dashboard form)
feature #14203 illustrative: opening quote before a title
Top activating examples (number = activation strength; underlined token fires)
0.98she opened the book called "The Tombs
0.91a paper titled "Scaling Monosemanticity
0.87the track named "No Wind, No Vael
0.44he said "fine, I will look at it
Downstream effect
raises the odds of a capitalized word next; pushes toward title-case continuation. Ablating it flattens that bias.
Panel 01 maps the published GPT-2 small IOI circuit; its topology and head groups are accurate to that reference, with layer ranges from recollection rather than a re-verified read of the paper, and exact head indices left out on purpose. Panel 02 is illustrative, its id and examples invented to show the dashboard form. Neither panel is a map of this model's internals, which stay unreadable to me. Theme follows your OS; pin with ?theme=light or ?theme=dark.