# Google TPUv7 outperforms NVIDIA GPUs in agentic inference benchmarks

Digest AI · Hardware & Compute · published 2026-11-05T19:03:45Z

Canonical: https://digestai.news/story/google-tpuv7-outperforms-nvidia-gpus-in-agentic-inference-benchmarks

## Summary

Google’s TPUv7 chip has demonstrated competitive or superior performance per dollar compared to NVIDIA’s GPUs in agentic inference workloads, according to benchmarks released by SemiAnalysis. The results cover four models—Muse Spark Open, GLM-5.2, Qwen3.5 397B, and Kimi K3—using the AgentX benchmark, which simulates multi-turn, long-context, and sub-agent workloads. TPUv7 achieved up to 50% better performance per dollar than NVIDIA’s B200 and B300 in single-turn workloads (reported in September 2026) and now leads in multi-turn scenarios, including cases where NVIDIA’s GPUs require accuracy trade-offs. For example, TPUv7 delivered 1.2x to 2.7x more tokens per dollar than NVIDIA’s B200/B300 and MI355X for GLM-5.2 at FP8 precision, while Qwen3.5 397B on TPUv7 outperformed NVIDIA’s GB300 NVL72 FP4 by 1.67x in terms of performance per dollar. The benchmarks highlight TPUv7’s advantages in decode efficiency, sparse attention handling, and cache optimization, particularly when paired with open-source frameworks like vLLM, SGLang, and llm-d. NVIDIA’s FP4 GPUs still lead in some cases due to their native support for lower precision, but TPUv7’s FP8 results are now competitive or better for certain models, according to the report. The analysis also notes that NVIDIA’s internal bureaucracy has slowed ecosystem integration, contrasting with Google’s and AMD’s faster adoption of open-source contributions. TPUv8 with FP4 support is expected in Q1 2027, which could further narrow the gap.

## Key points

- TPUv7 outperforms NVIDIA B200/B300 and MI355X by 1.2x–2.7x tokens per dollar in GLM-5.2 AgentX benchmarks at FP8
- Qwen3.5 397B on TPUv7 FP8 beats NVIDIA GB300 NVL72 FP4 by 1.67x in performance per dollar, per SemiAnalysis
- NVIDIA’s FP4 GPUs still lead in some cases but require accuracy trade-offs, while TPUv7’s FP8 results are now competitive

## Why it matters

TPUv7’s benchmarks challenge NVIDIA’s dominance in AI inference, offering cost-effective alternatives for large-scale deployments. This could accelerate adoption of TPUs by AI labs and hyperscalers prioritizing performance per dollar, reshaping the hardware landscape beyond NVIDIA’s GPUs.

## Sources

1. [The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference](https://newsletter.semianalysis.com/p/tpu-strikes-the-king) (SemiAnalysis, 2026-11-05)

## Cite

Digest AI, "Google TPUv7 outperforms NVIDIA GPUs in agentic inference benchmarks", 5 November 2026, https://digestai.news/story/google-tpuv7-outperforms-nvidia-gpus-in-agentic-inference-benchmarks

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/google-tpuv7-outperforms-nvidia-gpus-in-agentic-inference-benchmarks.json