{"version":1,"type":"story","url":"https://digestai.news/story/microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci","json":"https://digestai.news/story/microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci.json","markdown":"https://digestai.news/story/microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci.md","slug":"microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci","headline":"Microsoft and CMU host workshop on AI agent evaluation measurement science","summary":"Microsoft Research and Carnegie Mellon University's AI Measurement Science & Engineering Center (AIMSEC) will hold a two‑day workshop in New York on October 22‑23, 2026. The event aims to bring together more than 150 stakeholders from academia, industry, civil society and government to discuss a common \"measurement science\" for evaluating and governing AI agents.\n\nThe agenda separates evaluation into \"horizontal\" challenges that apply across agents and \"vertical\" use‑case‑specific criteria such as healthcare, software development and security. Breakout sessions will address long‑running tasks, multi‑agent collaboration, safety, red‑team testing, and how humans can intervene, understand, and recover from failures. Microsoft Research says the meeting will culminate in public documentation that records unresolved issues, points of consensus and gaps in current methods.\n\nThe discussion emphasizes that a single success‑rate score is insufficient; instead, the process, human burden, and controllability must be measured. Ultimately, humans will set the passing line for each application, using the evaluation results as material for responsible decision‑making.","keyPoints":["Microsoft Research and CMU AIMSEC host a two‑day workshop on Oct 22‑23, 2026 in New York.","Over 150 stakeholders will discuss horizontal and vertical evaluation dimensions for AI agents.","Public documentation of consensus, disagreements and open issues will be released after the meeting."],"whyItMatters":"Establishing shared standards for AI agent evaluation can improve safety, accountability and trust across industries that increasingly rely on autonomous tools.","category":{"slug":"policy","name":"Policy & Regulation","url":"https://digestai.news/category/policy"},"entities":{"companies":["Microsoft","Carnegie Mellon University"],"models":[],"people":[]},"firstPublishedAt":"2026-10-24T02:59:00Z","updatedAt":"2026-10-24T02:59:00Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"note.com","title":"What Score Makes an AI Agent Pass? The \"Measurement Science\" Advanced by Microsoft and CMU","url":"https://note.com/turuto/n/ne71184cfcccb?hl=en","publishedAt":"2026-10-24T02:59:00Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Microsoft and CMU host workshop on AI agent evaluation measurement science\", 24 October 2026, https://digestai.news/story/microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci","publisher":"Digest AI","title":"Microsoft and CMU host workshop on AI agent evaluation measurement science","datePublished":"2026-10-24T02:59:00Z","url":"https://digestai.news/story/microsoft-and-cmu-host-workshop-on-ai-agent-evaluation-measurement-sci"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}