Study finds retrieval, not reading, limits RAG in long-context QA
Researchers introduced evidence-based generation (EBG) to diagnose why retrieval-augmented generation (RAG) struggles with long-context question answering. The study argues that the bottleneck is not the model's ability to read long texts, but the RAG pipeline's inability to retrieve all necessary evidence chunks. By requiring models to provide line-level citations, the team created a joint…
Key points
- EBG metric scores answer correctness and grounding simultaneously using line-level citations.
- Similarity retrieval fails to surface all required chunks for multi-fact queries in RAG.
- Chunking destroys document meta-information needed to resolve cross-passage references.
The team reformulated existing corpora into new benchmarks, including MultiHop-RAG and QASPER-MP, and added synthetic datasets ranging from 10K to 1M characters. They tested ten proprietary frontier models and three open-weight large language models. The results show that similarity-based retrieval often fails to surface all required chunks for multi-fact queries, whereas long-context models can handle these tasks without chunking. Additionally, chunking destroys document-level meta-information needed to resolve references across passages.
The study highlights that standard answer-only evaluations miss significant grounding failures, such as incomplete citations in complex settings. When context contradicts a model's internal knowledge, the gap between simple answer accuracy and grounded accuracy widens. The authors have released the data and code for this work on GitHub to support further research into evidence-grounded evaluation.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- OpenAI brings back Aleksander Madry to focus on AI safety · 2 src
- OpenAI claims AI-generated proof of quasi‑Riemann hypothesis · 2 src
- Nvidia's Astranomy study reaches 74.8% success in robot control with GPT-6 Astra · 1 src
- PHP patches critical RCE flaws discovered with GPT-5.6 Sol, affecting up to 70% of websites · 8 src
- Ggml-org adds fused top‑k mask operation to llama.cpp · 1 src
Comments
via GitHub Discussions