Researchers introduce Context language models that manage their own Context
A new class of language models called Context Language Models (CLMs) lets the model treat its context as a mutable file and update it directly. This design enables the model to learn which information to retain and supports multiple agents each with separate context files.
Key points
- CLMs treat context as a mutable file, allowing models to update it directly.
- CLMs achieve 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp‑Plus versus SOTA.
- Online RL improves Qwen3.5‑9B on BrowseComp‑Plus by 47.6% using 12% fewer FLOPs.
Zero‑shot experiments with existing models show CLMs beating current context‑management baselines: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp‑Plus, 5% higher scores with 59% fewer FLOPs on a 12‑hour EdgeBench run, and a 65% improvement on a 24‑hour multi‑repository agent‑swarm task using the same compute. CLMs can also be steered with natural‑language instructions, raising held‑out accuracy by up to 35.9 points, and an online reinforcement‑learning method lifts Qwen3.5‑9B performance on BrowseComp‑Plus by 47.6% while cutting FLOPs by 12%. A co‑designed Suffix Cache Reuse layer further trims server‑side compute by 35% compared with standard SGLang at matched performance.
The story so far
6 episodes →- Researchers introduce Context language models that manage their own Contextthis story
Context Language Models
arxiv.org · 1 October 2026
Loading the full article…
This text was published by arxiv.org. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
4sources- Hacker News discussion · 44 pointsnews.ycombinator.com
- 5 Proven Techniques for Token Compression and Prompt OptimizationPress · KDnuggets ·
- Knowing When to Yield: Grounded Arbitration of User Corrections in Text-Based Embodied AgentsPrimary source · arXiv cs.AI ·
- Measuring the Microtask Eligibility Gap: When Is an Off-the-Shelf SLM Enough for an Agent Harness?Primary source · arXiv cs.AI ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic's Claude begins 'dreaming' to reorganize memory in tests · 1 src
- John Langford quits Microsoft, declares AI Frontiers over · 1 src
- Anthropic releases study on Claude usage in scientific research · 3 src
- MIT and BASF release CheMeleon for low-data molecular prediction · 2 src
- Anthropic tests Claude agents in EVE Online trading and sabotage detection · 1 src
Comments
via GitHub Discussions