Researchers introduce ReScopeBench to test long-lived AI agents’ experience retention
A new benchmark called ReScopeBench evaluates how AI agents manage experience over time in changing environments. The paper, posted on arXiv, focuses on agents that assist users across multiple tasks rather than single interactions. It distinguishes between structural knowledge (how an environment works) and rule-based knowledge (principles guiding decisions).
Key points
- ReScopeBench tests 18 interactive environments to measure how agents retain, revise, or lose experience
- 20 models tested show retained experience improves task performance but fails to adapt to changes
- Researchers separate structural knowledge (environment organization) from rule-based knowledge (task principles)
The authors propose an experience-first approach to build task streams around shared learning, then track how agents retain, revise, or lose experience. Across 20 models, they find retained experience boosts performance but agents fail to update it when environments change. Even with correct experience, tasks often remain unsolved, highlighting gaps in selective maintenance and effective use.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- OpenAI releases LifeVal evaluation for AI task performance · 3 src
- Perplexity adds live web searches to Decisions API for real-time answers · 1 src
- Suno adds MIDI file uploads for Pro and Premier users on Create · 1 src
- Redis adds agent routing, observability and messaging to Iris · 1 src
- Phaze raises $8.75M seed to let AI agents access company workstations remotely · 1 src
Comments
via GitHub Discussions