SemiAnalysis releases GDPx benchmark for frontier AI models
SemiAnalysis has launched GDPx, a new benchmark designed to measure the general agentic capabilities of frontier AI models. The benchmark focuses on high-value domains such as chip design, cybersecurity, biology, finance, legal, and software engineering, arguing that these sectors will drive the first trillion dollars of AI revenue. Unlike previous benchmarks like OpenAI's GDPval, which relied…
Key points
- SemiAnalysis launched GDPx, a benchmark covering 11 high-value domains like chip design and finance.
- The best model scored 12% on GDPx, with OpenAI and Anthropic models leading the rankings.
- GDPx uses realistic environments from 14 partners, unlike static benchmarks like OpenAI's GDPval.
The results show that even the best-performing models score only 12% on the full benchmark, indicating significant room for improvement. OpenAI and Anthropic models generally lead the rankings, with Opus 5.5 and GPT-6 Astra performing strongly, though their scores are sometimes lowered by safety classifiers. Gemini 4 Carbon is noted as a strong competitor, particularly in biology and software engineering. The authors highlight that safety classifiers significantly impact scores in domains like cyber and alignment, and that current models struggle heavily with complex finance and legal tasks, where many scored zero.
SemiAnalysis argues that existing benchmarks are inadequate for evaluating today's agents because they lack realistic work environments and often contain unfair grading criteria. By partnering with domain experts, GDPx aims to provide a more accurate picture of where frontier models can actually be deployed at high ROI. The benchmark is available for public exploration, and the authors suggest that tracking progress in these specific high-impact domains is more useful than broad GDP-based metrics.
Model pages: Opus 5.5 → · GPT-6 Astra → · Gemini 4 Carbon →
The story so far
6 episodes →- SemiAnalysis releases GDPx benchmark for frontier AI modelsthis story
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Mathematician Frank Calegari urges field to embrace AI as tool for broader exploration · 1 src
- Anthropic study finds AI‑assisted coding scores 17 points lower than manual coding · 1 src
- Anthropic research shows AI monitoring misses malicious training data that looks harmless · 5 src
- Study finds empathetic prompts push AI chatbots into trauma-like narratives · 1 src
- VicOne reports 28 AI‑discovered zero‑day vulnerabilities in robotics tools · 1 src
Comments
via GitHub Discussions