Researchers release OpenFrontiers benchmark with 12,151 open problems and 200 math tasks
A team of researchers has published OpenFrontiers, a new resource that pairs a corpus of 12,151 open research problems across disciplines with a benchmark of 200 mathematics problems. The benchmark includes 110 problems resolved in 2026 and 90 that remain open, each formalized in Lean 4 and verified by automated probes and two independent reviewers. A high-quality subset of 50 problems (HQ-50)…
Key points
- OpenFrontiers corpus contains 12,151 open research problems with context and progress data
- Benchmark includes 200 math problems formalized in Lean 4, 110 resolved in 2026 and 90 still open
- Seven LLM agents solved at most 7 of 40 resolved HQ-50 problems in closed-book evaluation
The evaluation found that closed-book rediscovery of solved problems was rare, with agents solving at most 7 of 40 resolved problems in the HQ-50 set. Success in the live-web condition largely reflected retrieval of already-published solutions rather than independent problem-solving. The dataset and benchmark are available on Hugging Face under the FlippyDora collection.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- AI startups aim to automate formal software verification with $365M funding · 2 src
- Researchers find instruction exposure unlocks rhetorical patterns in LLMs · 1 src
- New guide introduces LLMs for scientists and professionals · 1 src
- Researchers introduce Seq@K to test language models with sequential feedback · 6 src
- Researchers propose rank-aware method for private listwise LLM alignment · 1 src
Comments
via GitHub Discussions