Researchers release MemU-Bench to test GUI agents on long user sessions
Researchers introduced MemU-Bench, a new benchmark for testing GUI agents on 118 vague tasks across 29 long-horizon user interactions. Unlike existing tests, it simulates real-world ambiguity where users forget past steps, forcing agents to reconstruct context from noisy histories. The benchmark includes Success Rate, Interaction Efficiency (IE), and Success weighted by Path Length (SPL) to…
Key points
- MemU-Bench tests GUI agents on **118 vague tasks** across **29 long user sessions** with forgotten context
- Top models like **Gemini-3.1-Pro** score **16.4% SPL** without user history, end-to-end agents under **5%**
- **AppMem** framework improves **GPT-5.5’s SPL by 51.2%**, from **15.7% to 66.9%**
The study found that top models like Gemini-3.1-Pro scored only 16.4% SPL without user history, while end-to-end GUI agents failed below 5%. To improve performance, the team proposed AppMem, a framework that organizes user history into a structured memory bank. AppMem boosted GPT-5.5’s SPL from 15.7% to 66.9%, a 51.2% increase, outperforming simpler summary methods by 9.3%. The work highlights gaps in current agents’ ability to handle fragmented, real-world tasks.
Coverage and discussion
2sources- MEMENTO: Learning Context Management for Long Context Processing AgentsPrimary source · arXiv cs.CL ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- Researchers build an AI agent to optimize online retail bids and discounts · 1 src
- Claude API ends Sonnet 4.5 and Opus 4.8 fast mode on November 30 · 2 src
- OpenAI releases LifeVal evaluation for AI task performance · 2 src
- The Letter Company releases beta of letterpress-1 coding agent · 1 src
- Y Combinator CEO Garry Tan says he spends up to $200,000 a year on AI tokens · 1 src
Comments
via GitHub Discussions