Google blocks AI agents scraping search results, report says
Google is taking steps to curb automated scraping of search engine results pages (SERPs), with AI agents driving much of the traffic. Cloudflare data shows AI agent web traffic surged 1,700% over the past year, now accounting for more than half of internet traffic. Some of this traffic involves hybrid human-agent visits, which may still benefit websites, but Google is responding aggressively to…
Key points
- AI agent traffic grew **1,700%** year-over-year, now over half of internet traffic, per Cloudflare
- Google blocks rank trackers and AI scraping to prevent reverse engineering of its search algorithm
- Sundar Pichai cited cost reductions for AI queries in Q2 earnings, but AI traffic risks latency
The shift comes as AI-driven search results remain computationally expensive for Google. Sundar Pichai noted in Q2 earnings that the company is reducing costs per AI query through optimizations like the Gemini 3.6 Flash and Gemini 3.5 Flash-Lite models. However, the surge in AI agent traffic risks latency issues, particularly for AI-generated responses. SEO expert Ryan Jones warns rank tracking may soon become unsustainable due to Google’s anti-scraping measures, though the company has not confirmed this as its primary motive.
The story so far
3 episodes →- Google blocks AI agents scraping search results, report saysthis story
Google Is Fighting SERP Tracking. AI Agents May Make It Worse via @sejournal, @martinibuster
Search Engine Journal · 5 October 2026
Loading the full article…
This text was published by Search Engine Journal and written by Roger Montti. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Marketing & Small Business
All →- User uses NotebookLM and Claude Code to audit PR labeling for affiliate SNS posts · 1 src
- Google's Mueller says AI crawlers access sitemaps and RSS feeds · 1 src
- Meta and Google may win AI as only 2% of US households pay for it · 1 src
- Google Gemini integrates apps for small business owners · 1 src
- TikTok adds AI shopping assistant and one-click checkout to For You feed · 2 src
Comments
via GitHub Discussions