Researchers introduce SURE-Act to improve Tool-Using agents
Tool-using large language model agents often must choose between acting autonomously, gathering more information, or escalating to a human, but existing data only show the outcomes of actions actually taken. To fill this gap, the authors created a counterfactual dataset in ToolSandbox that records 3,115 outcomes from 1,036 decision points, evaluating each possible legal action for task success,…
1 source primary source
Key points
- Researchers built a counterfactual dataset with 3,115 action outcomes from 1,036 decision points in ToolSandbox.
- SURE-Act improves offline utility by 0.0309 and overall score by up to 0.0575 for the Qwen3-4B agent.
- SURE-Act outperforms SAGE-Agent and Calibrate-Then-Act decision rules by 0.1506 and 0.1438 respectively.
Read the original at arXiv cs.AI · by Qisai Liu, Qingyue Jiao, Bangwei Guo, Wenxi Dai, Changyi Yang, Jiahe Liu, Dimitris N. Metaxas, Yiyu Shi, Zhanhong Jiang, Minghao Guo primary sourceOpen source ↗
Topics · follow one to build your own front page
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic reports AI agents spend 80% of session waiting for humans, remeasurement shows about 30% · 1 src
- CoDA enables rapid codebook development for domain-specific text annotation · 1 src
- Study probes if LMs understand variable roles via SAEs · 1 src
- OR-License benchmark tests LLM agents on real operations research tasks · 1 src
- Study finds most LLM agent decisions on prompt injections are set at read time · 1 src
Comments
via GitHub Discussions