Scoring Explanations Without Reading Them: Structural Shortcuts in Automated-Interpretability
A new study reveals that automated interpretability scores for sparse-autoencoder features can be calculated without actually reading the explanations. Researchers found that over 90% of activating examples are cropped with the center token, and a simple rule based on this information achieves high accuracy. The study also shows that scoring rules can vary across different evaluation data…
1 source primary source
Key points
- 91.3% of activating examples are cropped with the center token
- A simple rule based on this information achieves balanced accuracy of 0.979
- Scoring rules vary across different evaluation data pipelines
Read the original at arXiv cs.CL · by Xupeng Chen primary sourceOpen source ↗
Topics · follow one to build your own front page
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- AI startups aim to automate formal software verification with $365M funding · 2 src
- Researchers find instruction exposure unlocks rhetorical patterns in LLMs · 1 src
- New guide introduces LLMs for scientists and professionals · 1 src
- Researchers introduce Seq@K to test language models with sequential feedback · 6 src
- Researchers propose rank-aware method for private listwise LLM alignment · 1 src
Comments
via GitHub Discussions