Goodfire researcher challenges superposition theory in AI interpretability essay
Nick Cammarata, a researcher at Goodfire, argues in a new essay that neural networks are far simpler to interpret than previously believed. His work rejects the widely accepted superposition hypothesis, which claims neurons combine unrelated concepts due to limited capacity. Cammarata reinterprets classic examples, like a neuron in InceptionV1 responding to both cat faces and car fronts, as…
Key points
- Goodfire’s Nick Cammarata rejects superposition theory, calling neural networks ‘already clean’ in a new essay
- His analysis of Qwen3-32B finds **90%** of neuron poles have one main job, **7%** handle two related ones
- OpenAI’s Roon calls it a historic moment, while Google DeepMind’s Neel Nanda calls it interesting but not a breakthrough
Cammarata’s findings suggest that in language models like Qwen3-32B, about 90 percent of neuron poles have one primary function, 7 percent handle two related concepts, and only 2 to 4 percent remain unexplained. He attributes past confusion to outdated architectures and weak analysis tools, advocating for automated bulk analysis over sparse autoencoders. OpenAI’s Roon praised the work as a potential historic moment, while Google DeepMind’s Neel Nanda cautioned against overhyping it, calling it a useful tool but not a breakthrough. Nanda still supports superposition and doubts small-model findings apply to larger systems. The essay’s technical companion paper has not yet been released.
Is AI interpretability close to a breakthrough? A new essay has researchers divided
The Decoder · 16 October 2026
Loading the full article…
This text was published by The Decoder and written by Maximilian Schreiner. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Nvidia pledges $250M in compute for UK AI agent safety research · 11 src
- Agentic RAG adds reasoning loops to retrieval systems for complex queries · 1 src
- OpenAI offers U.S. university labs free ChatGPT Business seats with Rosalind access · 3 src
- PhD students using AI in dissertations less likely to enter academia · 2 src
- KDnuggets lists five resources to learn about world models in AI · 1 src
Comments
via GitHub Discussions