{"version":1,"type":"story","url":"https://digestai.news/story/researchers-extract-circuits-from-language-models-using-attention-rout","json":"https://digestai.news/story/researchers-extract-circuits-from-language-models-using-attention-rout.json","markdown":"https://digestai.news/story/researchers-extract-circuits-from-language-models-using-attention-rout.md","slug":"researchers-extract-circuits-from-language-models-using-attention-rout","headline":"Researchers extract circuits from language models using Attention routing","summary":"The authors propose a method to extract circuits from language models by treating attention as a routing map from a single forward pass. They evaluate on GPT-2 Small, GPT-2 Medium, and Pythia-410M using 100 prompts per cell. Ablation of extracted edges hurts performance more than random edges. The extraction step costs one forward pass, whereas a head‑by‑head patch sweep costs about two orders of magnitude more. Code is available on GitHub.","keyPoints":["Method extracts circuits using attention routing from a single forward pass","Evaluation on GPT-2 Small, GPT-2 Medium, Pythia-410M with 100 prompts per cell","Extraction costs one forward pass, patch sweep costs ~100x more"],"whyItMatters":"Provides a cheap way to identify causal circuits in language models, enabling more efficient debugging and interpretability.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["GPT-2 Small","GPT-2 Medium","Pythia-410M"],"people":[]},"firstPublishedAt":"2026-09-23T04:00:00Z","updatedAt":"2026-09-23T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass","url":"https://arxiv.org/abs/2609.25285","publishedAt":"2026-09-23T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers extract circuits from language models using Attention routing\", 23 September 2026, https://digestai.news/story/researchers-extract-circuits-from-language-models-using-attention-rout","publisher":"Digest AI","title":"Researchers extract circuits from language models using Attention routing","datePublished":"2026-09-23T04:00:00Z","url":"https://digestai.news/story/researchers-extract-circuits-from-language-models-using-attention-rout"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}