# AdaMem improves soft compression for retrieval-augmented generation

Digest AI · Research · published 2026-09-22T04:00:00Z

Canonical: https://digestai.news/story/adamem-improves-soft-compression-for-retrieval-augmented-generation

## Summary

AdaMem is a new relevance‑guided soft‑compression framework for retrieval‑augmented generation (RAG). It maps passage‑relevance estimates to a query‑dependent allocation of a fixed memory‑token budget, allowing the model to give more memory tokens to higher‑scoring passages and omit low‑scoring ones.

Across six open‑domain QA benchmarks, AdaMem consistently outperforms the OSCAR baseline, which uses uniform allocation. Under standard 16× compression, AdaMem improves sub‑string match by up to 3.2 points (5.5%) over the uniform baseline, with an average relative gain of 3.4%. With aggressive 64× compression, the average relative gain rises to 14.6%, and the maximum improvement reaches 9.8 points (19.7%) on PopQA. AdaMem also matches the answer quality of the uncompressed model while achieving up to 4× lower inference latency than full‑context inference.

The work demonstrates that relevance‑guided memory allocation is especially effective when retrieval pools are large and memory budgets are tight, offering a more efficient way to deploy large language models with limited resources.

## Key points

- AdaMem allocates memory tokens based on passage relevance, improving compression efficiency.
- AdaMem outperforms OSCAR by up to 3.2 points under 16× compression.
- Under 64× compression, AdaMem gains 14.6% relative improvement, 9.8 points on PopQA.

## Why it matters

AdaMem’s relevance‑guided allocation reduces inference latency and boosts answer quality in retrieval‑augmented generation, enabling more efficient large‑scale language‑model deployments with limited memory budgets.

## Sources

1. [AdaMem: Adaptive Memory Token Allocation for Soft Compression in Retrieval-Augmented Generation](https://arxiv.org/abs/2609.22100) (arXiv cs.CL, 2026-09-22, primary source)

## Cite

Digest AI, "AdaMem improves soft compression for retrieval-augmented generation", 22 September 2026, https://digestai.news/story/adamem-improves-soft-compression-for-retrieval-augmented-generation

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/adamem-improves-soft-compression-for-retrieval-augmented-generation.json