DigestAI news desk

Cut through the AI noise.

Research

SemiAnalysis releases GDPx benchmark for frontier AI models

SemiAnalysis has launched GDPx, a new benchmark designed to measure the general agentic capabilities of frontier AI models. The benchmark focuses on high-value domains such as chip design, cybersecurity, biology, finance, legal, and software engineering, arguing that these sectors will drive the first trillion dollars of AI revenue. Unlike previous benchmarks like OpenAI's GDPval, which relied…

1 source

Key points

  • SemiAnalysis launched GDPx, a benchmark covering 11 high-value domains like chip design and finance.
  • The best model scored 12% on GDPx, with OpenAI and Anthropic models leading the rankings.
  • GDPx uses realistic environments from 14 partners, unlike static benchmarks like OpenAI's GDPval.

The results show that even the best-performing models score only 12% on the full benchmark, indicating significant room for improvement. OpenAI and Anthropic models generally lead the rankings, with Opus 5.5 and GPT-6 Astra performing strongly, though their scores are sometimes lowered by safety classifiers. Gemini 4 Carbon is noted as a strong competitor, particularly in biology and software engineering. The authors highlight that safety classifiers significantly impact scores in domains like cyber and alignment, and that current models struggle heavily with complex finance and legal tasks, where many scored zero.

SemiAnalysis argues that existing benchmarks are inadequate for evaluating today's agents because they lack realistic work environments and often contain unfair grading criteria. By partnering with domain experts, GDPx aims to provide a more accurate picture of where frontier models can actually be deployed at high ROI. The benchmark is available for public exploration, and the authors suggest that tracking progress in these specific high-impact domains is more useful than broad GDP-based metrics.

Model pages: Opus 5.5 → · GPT-6 Astra → · Gemini 4 Carbon →

The story so far

6 episodes →
  1. SemiAnalysis releases GDPx benchmark for frontier AI modelsthis story
Read the original at SemiAnalysis · by Max KanOpen source ↗
Topics · follow one to build your own front page

The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.

Comments

via GitHub Discussions

More in Research

All →

Related stories