{"version":1,"type":"story","url":"https://digestai.news/story/new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie","json":"https://digestai.news/story/new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie.json","markdown":"https://digestai.news/story/new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie.md","slug":"new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie","headline":"New York Times sues Brave over 30 billion webpages sold to AI companies","summary":"The New York Times filed a lawsuit against Brave on November 2, alleging the company copied more than 30 billion webpages without identifying its crawler or giving site owners a way to block it, then sold that content to AI companies including Anthropic, Perplexity, Mistral, xAI, and Cohere. Brave sells access to its search index through the Brave Search API, which its About page says is used by Cohere, Mistral, Perplexity, and You.com. The Times also claims copyright infringement and paywall circumvention. Brave has not yet responded to the suit.\n\nBrave's help page states its crawler \"does not advertise a differentiated user agent,\" meaning it has no unique name in robots.txt. Brave says it follows Googlebot's permissions, so sites cannot allow Google while blocking Brave. The only opt-out Brave offers is the noindex tag, which also removes pages from Google. A separate case was filed the same day by the Daily News, Chicago Tribune, and Orlando Sentinel. In July 2026, News Corp sued Brave over similar crawler concealment claims, per Semafor.","keyPoints":["NYT alleges Brave copied 30+ billion pages without a named crawler or opt-out mechanism","Brave Search API sells index access to Anthropic, Perplexity, Mistral, xAI, Cohere","Brave's crawler follows Googlebot rules; sites must use noindex to opt out, losing Google too"],"whyItMatters":"The case tests whether search indexes can be sold to AI firms without publisher consent or crawler transparency, affecting every site in Brave's 30B-page index.","category":{"slug":"policy","name":"Policy & Regulation","url":"https://digestai.news/category/policy"},"entities":{"companies":["Brave","New York Times","Anthropic","Perplexity","Mistral","Cohere"],"models":["Claude 2"],"people":["Simon Willison"]},"firstPublishedAt":"2026-11-06T20:00:19Z","updatedAt":"2026-11-06T20:00:19Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"Search Engine Journal","title":"Brave Sells Its Search Index To AI, And Your Website Is Probably In It via @sejournal, @slobodanmanic","url":"https://searchenginejournal.com/brave-sells-its-search-index-to-ai-and-your-website-is-probably-in-it/595829","publishedAt":"2026-11-06T20:00:19Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"Publishers vs Brave AI Content Scraping","url":"https://digestai.news/thread/new-york-times-sues-brave-over-ai-scraping-and-paywall-bypass","storyCount":2},"cite":{"text":"Digest AI, \"New York Times sues Brave over 30 billion webpages sold to AI companies\", 6 November 2026, https://digestai.news/story/new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie","publisher":"Digest AI","title":"New York Times sues Brave over 30 billion webpages sold to AI companies","datePublished":"2026-11-06T20:00:19Z","url":"https://digestai.news/story/new-york-times-sues-brave-over-30-billion-webpages-sold-to-ai-companie"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}