# Anthropic outlines third-party AI evaluation limits in new safety framework

Digest AI · Policy & Regulation · published 2026-10-19T10:33:00Z

Canonical: https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram

## Summary

Anthropic announced on October 15, 2026, that it granted six staff members from **METR**, a non-profit AI risk research group, access to internal systems and equipment comparable to its own risk assessment team. The scope excludes customer data, legally protected information, and model architecture details. **METR** can publish risk analyses and problem cases, but **Anthropic** retains final authority over deletions of confidential information, raising questions about evaluation independence.

The company’s new three-tiered system for access and publication rights highlights trade-offs between transparency and confidentiality. **Anthropic** emphasizes that external evaluation does not guarantee safety, especially when risks depend on usage environments beyond the model itself. The framework’s effectiveness remains untested, as future reports and operational records will determine how thoroughly risks are disclosed and how deletion requests are handled.

## Key points

- Anthropic gave **METR** access to internal systems but excluded customer data, legal protections, and model architecture
- **METR** can publish risk findings but **Anthropic** controls deletions of confidential details, limiting transparency
- The new framework does not prove safety—risks depend on how companies integrate AI in their own workflows

## Why it matters

Companies adopting AI must scrutinize evaluation frameworks beyond superficial endorsements. This framework shows how trade-offs between transparency and confidentiality shape trust in third-party assessments, affecting procurement decisions and risk management.

## Sources

1. [How reliable is third-party AI evaluation? Three points to check based on Anthropic's safety assessment](https://note.com/ai_mikatalab/n/n306212bcc55d?hl=en) (note.com, 2026-10-19)

## Cite

Digest AI, "Anthropic outlines third-party AI evaluation limits in new safety framework", 19 October 2026, https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/anthropic-outlines-third-party-ai-evaluation-limits-in-new-safety-fram.json