{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task","json":"https://digestai.news/story/researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task.json","markdown":"https://digestai.news/story/researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task.md","slug":"researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task","headline":"Researchers release SlackBench to test AI agents on real teamwork tasks","summary":"A team of researchers has introduced SlackBench, a new benchmark designed to evaluate AI agents on collaborative work tasks grounded in real-world scenarios. Unlike existing benchmarks that rely on synthetic or isolated tasks, SlackBench simulates complex, multi-party workflows where agents must synthesize information from conversations, code repositories, and evolving decisions—such as those in academic research projects. The dataset includes 140 queries spanning tasks like agentic search, goal-directed summarization, and reasoning across long conversations and GitHub repositories. It also tests agents’ ability to abstain from answering unanswerable questions or those based on false premises.\n\nThe benchmark tests 33 frontier models and agent frameworks, with the top-performing system scoring around 80% and most scoring below 66%. This highlights significant room for improvement in AI agents’ ability to handle realistic collaborative environments. SlackBench is available on Hugging Face and includes privacy-preserving reenactments of real research projects, providing a reproducible framework for further development.","keyPoints":["SlackBench tests AI agents on 140 queries simulating real teamwork tasks across conversations, code, and evolving decisions","Top-performing system scores 80%, but most agents score below 66%, showing gaps in collaborative reasoning","Dataset includes privacy-preserving reenactments of real academic research projects with GitHub and Google Docs data"],"whyItMatters":"SlackBench provides a more realistic test for AI agents in teamwork settings, pushing developers to improve multi-source reasoning and context retention—critical for workplace tools.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Hugging Face"],"models":[],"people":["Huanzhi Mao"]},"firstPublishedAt":"2026-11-05T05:00:00Z","updatedAt":"2026-11-05T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories","url":"https://arxiv.org/abs/2611.05512","publishedAt":"2026-11-05T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"New Benchmarks Test AI Math And Teamwork","url":"https://digestai.news/thread/researchers-release-openfrontiers-benchmark-with-12-151-open-problems-and-200","storyCount":2},"cite":{"text":"Digest AI, \"Researchers release SlackBench to test AI agents on real teamwork tasks\", 5 November 2026, https://digestai.news/story/researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task","publisher":"Digest AI","title":"Researchers release SlackBench to test AI agents on real teamwork tasks","datePublished":"2026-11-05T05:00:00Z","url":"https://digestai.news/story/researchers-release-slackbench-to-test-ai-agents-on-real-teamwork-task"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}