AdaMem improves soft compression for retrieval-augmented generation
AdaMem is a new relevance‑guided soft‑compression framework for retrieval‑augmented generation (RAG). It maps passage‑relevance estimates to a query‑dependent allocation of a fixed memory‑token budget, allowing the model to give more memory tokens to higher‑scoring passages and omit low‑scoring ones.
Key points
- AdaMem allocates memory tokens based on passage relevance, improving compression efficiency.
- AdaMem outperforms OSCAR by up to 3.2 points under 16× compression.
- Under 64× compression, AdaMem gains 14.6% relative improvement, 9.8 points on PopQA.
Across six open‑domain QA benchmarks, AdaMem consistently outperforms the OSCAR baseline, which uses uniform allocation. Under standard 16× compression, AdaMem improves sub‑string match by up to 3.2 points (5.5%) over the uniform baseline, with an average relative gain of 3.4%. With aggressive 64× compression, the average relative gain rises to 14.6%, and the maximum improvement reaches 9.8 points (19.7%) on PopQA. AdaMem also matches the answer quality of the uncompressed model while achieving up to 4× lower inference latency than full‑context inference.
The work demonstrates that relevance‑guided memory allocation is especially effective when retrieval pools are large and memory budgets are tight, offering a more efficient way to deploy large language models with limited resources.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- Researchers propose zero-inference prospective term that boosts personal memory recall · 1 src
- PsyAgentBench tests if LLMs truly mimic human psychological biases · 1 src
- Indy Autonomous Challenge features first road-course overtakes, Unimore wins · 1 src
- Researchers unveil TangleDiff to design entangled protein hydrogels · 1 src
- Anthropic reports Claude leads 26% of its AI research and development tasks · 3 src
Comments
via GitHub Discussions