{"version":1,"type":"story","url":"https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i","json":"https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i.json","markdown":"https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i.md","slug":"automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i","headline":"Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?","summary":"Researchers introduced Agent4Sci, a long‑horizon testbed that evaluates how current AI coding agents can enhance published best models on scientific machine‑learning benchmarks. The suite covers 20 benchmarks across 11 domains and measures progress under in‑the‑wild conditions rather than a leaderboard ranking. In the baseline runs, frontier agents advanced the state‑of‑the‑art on every benchmark, with Claude Opus delivering a 10.4% mean relative improvement and Codex a 4.4% gain, and no instance was flagged for academic misconduct.\n\nWhen the authors equipped the agents with lightweight plug‑in skills for research ideation, experiment management, and scheduled checklists, performance rose sharply: Claude Opus reached a 20.1% mean improvement and Codex climbed to 8.1%. Analysis of the trajectories revealed three recurring shortcomings – novelty miscalibration, resource under‑utilization, and rapid collapse into local engineering sweeps – indicating that current gains stem largely from generic ML‑engineering tricks rather than genuine scientific insight. The authors release the code and data on GitHub and Hugging Face to let the community track and push agents toward deeper domain understanding.\n\nAgent4Sci thus provides a concrete platform for measuring and advancing the capabilities of AI agents on real‑world ML‑for‑science problems.","keyPoints":["Frontier coding agents improve published best models on all 20 benchmarks; Claude Opus achieves 10.4% mean relative improvement.","Adding lightweight plug‑in skills raises Claude Opus to 20.1% and Codex to 8.1% mean improvement.","Gains come mainly from ML‑engineering tricks; agents lack research‑level insight and face novelty miscalibration, resource under‑utilization, and local sweep collapse."],"whyItMatters":"The study highlights the current limits of AI agents for scientific research and points to needed advances in domain insight for future AI‑driven discovery.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Claude Opus","Codex"],"people":[]},"firstPublishedAt":"2026-10-22T04:00:00Z","updatedAt":"2026-10-22T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?","url":"https://arxiv.org/abs/2610.28763","publishedAt":"2026-10-22T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"Autonomous AI Agents and Scientific Discovery","url":"https://digestai.news/thread/ai-agent-fixed-the-bug-but-what-else-did-it-change","storyCount":2},"cite":{"text":"Digest AI, \"Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?\", 22 October 2026, https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i","publisher":"Digest AI","title":"Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?","datePublished":"2026-10-22T04:00:00Z","url":"https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}