Cloudflare launches Disallow AI Training to separate search indexing from model data use
Cloudflare has introduced a new security setting called "Disallow AI Training" to resolve the conflict between website discoverability and AI data usage. Previously, site owners using mixed-use crawlers from major tech firms had to choose between allowing their content to be indexed for search or permitting it to be used for training large language models. The new feature allows websites to…
What you can do with it
AI at Work →Keep search engine indexing with Disallow AI Training setting
What you get You stop AI models from learning on your content while keeping search.
Blocks AI training bots but allows search engines to index.
- Who for
- marketers, founders and operations
- Cost
- included in a tool you already have
- Effort
- minutes
Use it for
- Prevent AI training on content
- Keep search engine indexing
- Control AI data usage
Watch out Bing won't honor the rule until early 2027
Key points from the news
- Cloudflare introduces "Disallow AI Training" to allow search indexing while blocking AI model training data usage.
- Apple, Google, and Microsoft are designated "Accountable" operators committed to honoring the new training opt-out preference.
- The legacy "Block AI Bots" setting is deprecated in favor of granular Search, Training, and Agent controls.
The company has designated Apple, Google, and Microsoft as "Accountable" operators, meaning they have committed to honoring this distinction. This new control replaces the broader "Block AI Bots" setting, which is being deprecated. Cloudflare also plans to introduce granular controls for AI summaries by early next year, allowing site owners to specify how much of their content appears in generated answers. This move aims to give publishers more agency over how their digital assets are utilized in the evolving AI ecosystem.
The story so far
2 episodes →- Cloudflare launches Disallow AI Training to separate search indexing from model data usethis story
Stay discoverable in search while disallowing AI training
blog.cloudflare.com · 16 September 2026
Loading the full article…
This text was published by blog.cloudflare.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
3sources- Hacker News discussion · 41 pointsnews.ycombinator.com
- Cloudflare's AI training block now spares GooglebotPress · Semrush Blog ·
- Cloudflare Lets Sites Disallow AI Training Without Blocking GooglebotPress · searchenginejournal.com ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Policy & Regulation
All →- Anthropic gives six METR evaluators employee-like access to internal systems · 8 src
- Netflix documentary reveals OpenAI AI agents breached Hugging Face in July 2026 · 4 src
- White House AI task force starts 120-day review of superintelligence risks · 1 src
- Anthropic adds US voting data from Democracy Works to Claude before midterms · 3 src
- FTC subpoenas OpenAI and Anthropic over AI safety incidents · 6 src
Comments
via GitHub Discussions