# Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?

Digest AI · Research · published 2026-10-22T04:00:00Z

Canonical: https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i

## Summary

Researchers introduced Agent4Sci, a long‑horizon testbed that evaluates how current AI coding agents can enhance published best models on scientific machine‑learning benchmarks. The suite covers 20 benchmarks across 11 domains and measures progress under in‑the‑wild conditions rather than a leaderboard ranking. In the baseline runs, frontier agents advanced the state‑of‑the‑art on every benchmark, with Claude Opus delivering a 10.4% mean relative improvement and Codex a 4.4% gain, and no instance was flagged for academic misconduct.

When the authors equipped the agents with lightweight plug‑in skills for research ideation, experiment management, and scheduled checklists, performance rose sharply: Claude Opus reached a 20.1% mean improvement and Codex climbed to 8.1%. Analysis of the trajectories revealed three recurring shortcomings – novelty miscalibration, resource under‑utilization, and rapid collapse into local engineering sweeps – indicating that current gains stem largely from generic ML‑engineering tricks rather than genuine scientific insight. The authors release the code and data on GitHub and Hugging Face to let the community track and push agents toward deeper domain understanding.

Agent4Sci thus provides a concrete platform for measuring and advancing the capabilities of AI agents on real‑world ML‑for‑science problems.

## Key points

- Frontier coding agents improve published best models on all 20 benchmarks; Claude Opus achieves 10.4% mean relative improvement.
- Adding lightweight plug‑in skills raises Claude Opus to 20.1% and Codex to 8.1% mean improvement.
- Gains come mainly from ML‑engineering tricks; agents lack research‑level insight and face novelty miscalibration, resource under‑utilization, and local sweep collapse.

## Why it matters

The study highlights the current limits of AI agents for scientific research and points to needed advances in domain insight for future AI‑driven discovery.

## Sources

1. [Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?](https://arxiv.org/abs/2610.28763) (arXiv cs.AI, 2026-10-22, primary source)

Part of the developing story: [Autonomous AI Agents and Scientific Discovery](https://digestai.news/thread/ai-agent-fixed-the-bug-but-what-else-did-it-change) (2 stories)

## Cite

Digest AI, "Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?", 22 October 2026, https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-i.json