Researchers warn automatic scorers may mislead LLM evaluation studies
A new paper on arXiv argues that many LLM studies overlook how automatic scorers can distort results when comparing model responses to different prompts. The authors note that scorers often use a single overall accuracy or F1 score to validate their work, but this approach can mask prompt-specific errors. For example, a scorer might flag 2.6% to 33.3% of responses as incorrect when a primary…
Key points
- Six models tested across 180 invented-entity questions and six prompt conditions in the study
- Automatic scorers flagged 2.6%–33.3% of responses as incorrect when human raters disagreed
- Researchers say prompt-specific comparisons need separate validation to avoid misleading results
The study tested six models across 180 invented-entity questions and six prompt conditions. In one audit of 288 responses, the scorer’s F1 score was high, but uncertainty in prompt-specific estimates made conclusions unreliable. A second audit of 300 cases showed that an apparent 9.3-point difference in model performance shrank to 2.3 points when evaluated by human raters. The authors conclude that researchers should validate both prompts and scorer/rater uncertainty separately, rather than relying on aggregate metrics alone.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers release LAMB, a Latin language model trained on 74B tokens · 1 src
- Researchers explore how aligned LLMs generalize to new tasks · 1 src
- Researchers propose agentic framework for emotion reasoning with evidence checks · 1 src
- Researchers propose 24 quality dimensions for knowledge graphs used by LLMs · 1 src
- Researchers introduce Schema Coherence Inventory to test belief consistency in language models · 1 src
Comments
via GitHub Discussions