Self-Evolving agents show limited learning on Hidden-Rule benchmark
Researchers introduced CLR-Bench, a benchmark to test whether self-evolving agents can learn hidden rules from feedback alone. The benchmark uses chains where each level adds one hidden rule based on opaque labels representing named groupings like airline alliance members. Six recent self-evolution methods were evaluated, with the best gaining at most 5 points over a frozen agent and excluding…
1 source primary source
Key points
- CLR-Bench tests self-evolving agents on learning hidden rules from feedback
- Best methods gain at most 5 points over frozen agent, exclude 6-12% of unseen members
- With GPT-6-Sol, ACE and Dynamic Cheatsheet reach 78% accuracy and solve 59-60% of cells
Model page: GPT-6 Sol →
Read the original at arXiv cs.AI · by Chenjie Qiu, Boyan Xu, Shaobin Shi, Bingfeng Chen, Ruichu Cai, Dongsheng Luo primary sourceOpen source ↗
Topics · follow one to build your own front page
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- SemiAnalysis releases GDPx benchmark with Claude Opus 5.5 leading at 12% · 3 src
- Gemma-3-12b fails to detect misinformation when poisoned claims dominate retrieved evidence · 1 src
- Researchers release BILAL-BENCH to test Bengali AI models’ Islamic text fabrication · 1 src
- Researchers prune 729 attention heads to cut GPT-2-XL’s memorisation by 91.5% · 1 src
- arXiv paper finds shortcuts distort RAG benchmarking accuracy · 1 src
Comments
via GitHub Discussions