Researchers introduce Context language models that manage their own Context
A new class of language models called Context Language Models (CLMs) lets the model treat its context as a mutable file and update it directly. This design enables the model to learn which information to retain and supports multiple agents each with separate context files.
Key points
- CLMs treat context as a mutable file, allowing models to update it directly.
- CLMs achieve 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp‑Plus versus SOTA.
- Online RL improves Qwen3.5‑9B on BrowseComp‑Plus by 47.6% using 12% fewer FLOPs.
Zero‑shot experiments with existing models show CLMs beating current context‑management baselines: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp‑Plus, 5% higher scores with 59% fewer FLOPs on a 12‑hour EdgeBench run, and a 65% improvement on a 24‑hour multi‑repository agent‑swarm task using the same compute. CLMs can also be steered with natural‑language instructions, raising held‑out accuracy by up to 35.9 points, and an online reinforcement‑learning method lifts Qwen3.5‑9B performance on BrowseComp‑Plus by 47.6% while cutting FLOPs by 12%. A co‑designed Suffix Cache Reuse layer further trims server‑side compute by 35% compared with standard SGLang at matched performance.
Context Language Models
arxiv.org · 1 October 2026
Loading the full article…
This text was published by arxiv.org. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Hacker News discussion · 44 pointsnews.ycombinator.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- CellCarta and Imagene AI expand AI biomarker collaboration for pharma · 1 src
- University of Alabama model predicts violence risk from video, audio and motion · 1 src
- Hansa Biopharma adopts Cradle’s AI platform for autoimmune disease drug discovery · 1 src
- UC Irvine study finds ChatGPT use linked to 25% drop in exam scores · 1 src
- Qwen4Exp adds MTP support in llama.cpp pull request · 1 src
Comments
via GitHub Discussions