AIBuildAI-2.5 ranks first on MLE-Bench with 73.3% medal rate
AIBuildAI-2.5 is an autonomous agentic system designed to build AI models by treating model construction as a tree‑search problem. The authors identify three efficiency gaps in prior agents—limited candidate execution, lack of resource‑aware scheduling, and reliance on a single powerful LLM for all tasks—and propose solutions.
Key points
- AIBuildAI-2.5 introduces LLM‑guided tree search that scores candidates on expected improvement, grounding, and feasibility
- The system adds a resource‑aware scheduler and a router that delegates low‑cost LLMs to simple tasks
- It achieved a 73.3% medal rate, ranking first on MLE‑Bench and beating a strong baseline on six AIRS‑Bench tasks
The new system adds an LLM‑guided tree search where a judge scores each candidate on expected improvement, grounding, and feasibility, and a selector ranks the pool using those scores and the current search state. A scheduler launches training jobs based on real‑time hardware availability, while a router assigns cheaper LLMs to low‑complexity steps, reserving the strongest model for the hardest sub‑tasks. In benchmark evaluation, AIBuildAI-2.5 achieved a 73.3% medal rate, placing first on MLE‑Bench, and outperformed a strong baseline across six autonomous AI research tasks from AIRS‑Bench.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Study finds wiki-indexed LLM outperforms vector RAG for cross-unit questions in ML courses · 1 src
- Study proposes skill habits to fix AI agent inconsistency · 1 src
- Researchers propose latent equivalence learning for enterprise data agents, score 94.67% on benchmark · 1 src
- Researchers extract circuits from language models using Attention routing · 1 src
- ReAdapt improves warm‑introduction and reaction selection accuracy for Gemini‑3‑Flash · 1 src
Comments
via GitHub Discussions