Researchers introduce ResMem to turn retrieved knowledge into parametric memory
A new paper on arXiv presents ResMem, a method that lets a language model learn from retrieved text offline and store the result in a compact parametric memory. The authors say this avoids the storage and inference costs of keeping a live retrieval index such as a k-nearest-neighbor datastore.
Key points
- ResMem distills retrieved knowledge into a parametric memory, avoiding online retrieval at inference
- Storage for teacher construction drops 94.7% versus a kNN datastore on a Wikipedia subset
- Mean token F1 improves from 32.83% to 34.48% across five QA datasets
In their design, the LLM reads retrieved evidence alongside the input to produce teacher targets. A student model with the same backbone then learns an additive logit residual that aligns its predictions with the teacher, while updates stay confined to the parametric memory. On a Wikipedia subset, ResMem cuts the storage needed for teacher construction by 94.7% compared with a kNN pipeline. Across five question-answering datasets, mean token F1 rises from 32.83% for the frozen LLM to 34.48%, and the method breaks 21% fewer of the model's originally correct answers than a kNN-trained memory. Code and model weights are released on GitHub and Hugging Face.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Circuit Oracle automates attribution graph analysis · 1 src
- arXiv study finds AI humanizers alter 35–65% of fact units in documents · 1 src
- Researchers propose gavagai to improve Vision-Language model alignment · 1 src
- Gemini 3.6 Flash shows language-dependent negotiation behavior in study · 1 src
- Researchers outline framework for unifying embedding models across modalities · 1 src
Comments
via GitHub Discussions