{"version":1,"type":"story","url":"https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram","json":"https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram.json","markdown":"https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram.md","slug":"anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram","headline":"Anthropic outlines third-party AI evaluation limits in new safety framework","summary":"Anthropic announced on October 15, 2026, that it granted six staff members from **METR**, a non-profit AI risk research group, access to internal systems and equipment comparable to its own risk assessment team. The scope excludes customer data, legally protected information, and model architecture details. **METR** can publish risk analyses and problem cases, but **Anthropic** retains final authority over deletions of confidential information, raising questions about evaluation independence.\n\nThe company’s new three-tiered system for access and publication rights highlights trade-offs between transparency and confidentiality. **Anthropic** emphasizes that external evaluation does not guarantee safety, especially when risks depend on usage environments beyond the model itself. The framework’s effectiveness remains untested, as future reports and operational records will determine how thoroughly risks are disclosed and how deletion requests are handled.","keyPoints":["Anthropic gave **METR** access to internal systems but excluded customer data, legal protections, and model architecture","**METR** can publish risk findings but **Anthropic** controls deletions of confidential details, limiting transparency","The new framework does not prove safety—risks depend on how companies integrate AI in their own workflows"],"whyItMatters":"Companies adopting AI must scrutinize evaluation frameworks beyond superficial endorsements. This framework shows how trade-offs between transparency and confidentiality shape trust in third-party assessments, affecting procurement decisions and risk management.","category":{"slug":"policy","name":"Policy & Regulation","url":"https://digestai.news/category/policy"},"entities":{"companies":["Anthropic","METR"],"models":[],"people":["Niko Hiiragi"]},"firstPublishedAt":"2026-10-19T10:33:00Z","updatedAt":"2026-10-19T10:33:00Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"note.com","title":"How reliable is third-party AI evaluation? Three points to check based on Anthropic's safety assessment","url":"https://note.com/ai_mikatalab/n/n306212bcc55d?hl=en","publishedAt":"2026-10-19T10:33:00Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Anthropic outlines third-party AI evaluation limits in new safety framework\", 19 October 2026, https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram","publisher":"Digest AI","title":"Anthropic outlines third-party AI evaluation limits in new safety framework","datePublished":"2026-10-19T10:33:00Z","url":"https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}