{"version":1,"type":"story","url":"https://digestai.news/story/study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt","json":"https://digestai.news/story/study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt.json","markdown":"https://digestai.news/story/study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt.md","slug":"study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt","headline":"Study finds AI experts underestimated progress on benchmarks and adoption","summary":"A new interim report from the Forecasting Research Institute (FRI) reveals that top AI experts, including professors, researchers, and economists, consistently underestimated recent advancements in AI capabilities. The study, based on data from the LEAP (Longitudinal Expert AI Panel), surveyed 339 specialists—such as 30 professors from top-20 institutions, 10 of the 200 most-cited AI authors, and superforecasters—between mid-2022 and 2025. Among the findings, AI achieved gold-medal performance in the International Mathematical Olympiad in July 2025, five years ahead of the median expert forecast and ten years ahead of superforecasters’ predictions made in 2022. Similarly, AI may have solved a Millennium Prize Problem, though verification remains unclear. Experts also predicted AI models wouldn’t match virologists on troubleshooting benchmarks until 2030 or 2034, but FRI suggests this likely occurred by April 2025. Economic forecasts were similarly off: experts estimated the highest annual recurring revenue (ARR) for any AI company at $20 billion by the end of 2026, while FRI cites Anthropic’s ARR of roughly $100 billion in September 2026 as likely already surpassed.\n\nNot all forecasts were overly conservative. Biosecurity and virology experts’ predictions about AI-assisted lab tasks were closer to reality, though a small trial showed minimal impact from language models. Self-driving car forecasts were also revised downward, with experts’ 7.3% estimate for autonomous ride-hailing trips in 2027 contradicted by an LLM projection of 2.5%. FRI notes that forecasts on economic growth, employment, and AI harms remain unreliable. Meanwhile, respondents’ views on AI’s long-term significance have shifted upward, with the probability of it becoming a ‘technology of the century’ rising from 31% to 36% for experts and 28% to 35% for superforecasters over nine months. FRI plans to refine its methods, including tracking rapid-progress forecasters and comparing LLM projections with human predictions.","keyPoints":["FRI’s LEAP panel of 339 experts—including 30 top professors and 10 of the 200 most-cited AI researchers—underestimated AI progress on benchmarks like math and virology","AI achieved gold at the International Mathematical Olympiad in 2025, five years ahead of expert forecasts and ten years ahead of superforecasters’ 2022 predictions","Experts’ median ARR estimate for AI companies by 2026 was $20 billion, but FRI cites Anthropic’s $100B ARR in September 2026 as likely already exceeded"],"whyItMatters":"The study highlights systemic underestimation in AI forecasting, exposing gaps between expert predictions and rapid real-world progress. For policymakers and investors, it underscores the need for adaptive strategies as AI advances outpace traditional timelines. The findings also raise questions about how to better calibrate expectations in a field moving faster than anticipated.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Forecasting Research Institute","Anthropic"],"models":[],"people":[]},"firstPublishedAt":"2026-09-24T19:18:33Z","updatedAt":"2026-09-24T19:18:33Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"The Decoder","title":"Top AI experts badly underestimated how fast the field is moving, study finds","url":"https://the-decoder.com/top-ai-experts-badly-underestimated-how-fast-the-field-is-moving-study-finds","publishedAt":"2026-09-24T19:18:33Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Study finds AI experts underestimated progress on benchmarks and adoption\", 24 September 2026, https://digestai.news/story/study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt","publisher":"Digest AI","title":"Study finds AI experts underestimated progress on benchmarks and adoption","datePublished":"2026-09-24T19:18:33Z","url":"https://digestai.news/story/study-finds-ai-experts-underestimated-progress-on-benchmarks-and-adopt"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}