Developing story · 2 episodes · 30 Oct to 2 Nov
AI Companies Unite on Robots.txt Training Limits
The saga follows major AI developers collaborating to establish a unified robots.txt policy that prevents their models from scraping web content for training. After the latest episode, OpenAI, Google, Microsoft, and Anthropic have officially backed a shared opt‑out mechanism, solidifying the industry‑wide commitment.
-
OpenAI, Google, Microsoft propose common robots.txt rule to block AI training
OpenAI, Google, Microsoft, and Anthropic announced on October 30, 2026, a draft policy to let website owners signal whether their content can be used for AI training via an updated robots.txt file.…
1 source -
OpenAI, Google, Microsoft, and Anthropic back shared robots.txt opt-out for AI training
OpenAI, Google, Microsoft, and Anthropic have announced support for a shared robots.txt signal that allows publishers to opt out of AI training across multiple companies with a single rule. The…
13 sources primary source
Who and what