Researchers propose Sieve and Sage to filter RALM distractions for better abstention
A new approach called Sieve and Sage aims to improve how Retrieval-Augmented Language Models (RALMs) handle unreliable or conflicting evidence. The method splits retrieval failures into two categories: unanswerable queries (missing evidence) and distracted queries (mixed or conflicting information). Sieve, a lightweight module, first filters out distracting evidence before Sage, a more…
Key points
- Sieve and Sage decomposes retrieval failures into unanswerable and distracted states for better filtering
- Claims 69.4% accuracy improvement and 55.2% Macro-F1 gain over existing one-stage RALM methods
- Lightweight Sieve module reduces computational cost by up to 1.99x before Sage generates responses
The authors claim this two-step process boosts accuracy by up to 69.4 percentage points and Macro-F1 by 55.2 percentage points over one-stage baselines. It also speeds up the process by up to 1.99x. Tests were run across general and high-stakes expert domains, though no external validation is mentioned.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers release ArgGYM benchmark for testing defeasible reasoning in AI models · 1 src
- Researchers release SimTrace for generating synthetic user behavior data · 1 src
- MetaPersona framework uses 11,000+ studies to build synthetic populations for AI tasks · 1 src
- Researchers propose DLFP controller to cut AI inference latency by up to 30% · 1 src
- Researchers introduce GoldiMask to improve diffusion language model fine-tuning · 1 src
Comments
via GitHub Discussions