{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-openfrontiers-benchmark-with-12-151-open-problems","json":"https://digestai.news/story/researchers-release-openfrontiers-benchmark-with-12-151-open-problems.json","markdown":"https://digestai.news/story/researchers-release-openfrontiers-benchmark-with-12-151-open-problems.md","slug":"researchers-release-openfrontiers-benchmark-with-12-151-open-problems","headline":"Researchers release OpenFrontiers benchmark with 12,151 open problems and 200 math tasks","summary":"A team of researchers has published OpenFrontiers, a new resource that pairs a corpus of 12,151 open research problems across disciplines with a benchmark of 200 mathematics problems. The benchmark includes 110 problems resolved in 2026 and 90 that remain open, each formalized in Lean 4 and verified by automated probes and two independent reviewers. A high-quality subset of 50 problems (HQ-50) was evaluated using seven LLM agents under standardized compute budgets across three conditions: closed-book, frozen-retrieval, and live-web access.\n\nThe evaluation found that closed-book rediscovery of solved problems was rare, with agents solving at most 7 of 40 resolved problems in the HQ-50 set. Success in the live-web condition largely reflected retrieval of already-published solutions rather than independent problem-solving. The dataset and benchmark are available on Hugging Face under the FlippyDora collection.","keyPoints":["OpenFrontiers corpus contains 12,151 open research problems with context and progress data","Benchmark includes 200 math problems formalized in Lean 4, 110 resolved in 2026 and 90 still open","Seven LLM agents solved at most 7 of 40 resolved HQ-50 problems in closed-book evaluation"],"whyItMatters":"Provides a standardized way to measure whether LLMs can independently rediscover scientific solutions versus retrieving known answers, a key step toward AI-assisted research.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Hugging Face"],"models":[],"people":[]},"firstPublishedAt":"2026-10-30T04:00:00Z","updatedAt":"2026-10-30T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery","url":"https://arxiv.org/abs/2610.37739","publishedAt":"2026-10-30T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release OpenFrontiers benchmark with 12,151 open problems and 200 math tasks\", 30 October 2026, https://digestai.news/story/researchers-release-openfrontiers-benchmark-with-12-151-open-problems","publisher":"Digest AI","title":"Researchers release OpenFrontiers benchmark with 12,151 open problems and 200 math tasks","datePublished":"2026-10-30T04:00:00Z","url":"https://digestai.news/story/researchers-release-openfrontiers-benchmark-with-12-151-open-problems"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}