{"version":1,"type":"story","url":"https://digestai.news/story/self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark","json":"https://digestai.news/story/self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark.json","markdown":"https://digestai.news/story/self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark.md","slug":"self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark","headline":"Self-Evolving agents show limited learning on Hidden-Rule benchmark","summary":"Researchers introduced CLR-Bench, a benchmark to test whether self-evolving agents can learn hidden rules from feedback alone. The benchmark uses chains where each level adds one hidden rule based on opaque labels representing named groupings like airline alliance members. Six recent self-evolution methods were evaluated, with the best gaining at most 5 points over a frozen agent and excluding only 6-12% of unseen members. With a stronger model (GPT-6-Sol), ACE and Dynamic Cheatsheet methods learned from feedback alone, reaching 78% accuracy and solving 59-60% of single cells, surpassing raw history performance. A fixed lesson stating the naming convention improved performance by 14.5 to 16.5 points.","keyPoints":["CLR-Bench tests self-evolving agents on learning hidden rules from feedback","Best methods gain at most 5 points over frozen agent, exclude 6-12% of unseen members","With GPT-6-Sol, ACE and Dynamic Cheatsheet reach 78% accuracy and solve 59-60% of cells"],"whyItMatters":"The benchmark reveals current limits in agent self-evolution, showing most methods struggle to learn implicit rules without explicit instruction, highlighting a gap in adaptive AI capabilities.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["GPT-6-Sol"],"people":[]},"firstPublishedAt":"2026-11-02T05:00:00Z","updatedAt":"2026-11-02T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"More Rules, Longer Lists? Benchmarking Self-Evolving Agents on a Hidden-Rule Curriculum","url":"https://arxiv.org/abs/2610.40758","publishedAt":"2026-11-02T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Self-Evolving agents show limited learning on Hidden-Rule benchmark\", 2 November 2026, https://digestai.news/story/self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark","publisher":"Digest AI","title":"Self-Evolving agents show limited learning on Hidden-Rule benchmark","datePublished":"2026-11-02T05:00:00Z","url":"https://digestai.news/story/self-evolving-agents-show-limited-learning-on-hidden-rule-benchmark"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}