{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-vaguebench-benchmark-exposing-ambient-context-illu","json":"https://digestai.news/story/researchers-release-vaguebench-benchmark-exposing-ambient-context-illu.json","markdown":"https://digestai.news/story/researchers-release-vaguebench-benchmark-exposing-ambient-context-illu.md","slug":"researchers-release-vaguebench-benchmark-exposing-ambient-context-illu","headline":"Researchers release VagueBench benchmark exposing ambient context illusion in agents","summary":"The paper introduces VagueBench, a new benchmark built on the AI2‑THOR simulation to test how embodied agents handle instructions that lack non‑essential contextual cues. Using a human‑agent co‑generation pipeline, the authors created 1,303 instruction pairs covering entities, procedures, constraints and prior knowledge, and verified that the core task semantics remain unchanged.\n\nSix state‑of‑the‑art embodied agents were evaluated on VagueBench. Results show a relative 19.65% drop in task success rate and a 54.10% increase in empty‑plan rate, indicating strong dependence on auxiliary linguistic signals. The authors also demonstrate that the same ambient context illusion occurs in real‑world robotic systems, where missing contextual details can lead to incorrect decisions and potential safety risks.\n\nThe study highlights a previously overlooked robustness gap for agents deployed in open‑world settings, suggesting that future research must address context‑dependency to improve reliability outside controlled environments.","keyPoints":["VagueBench contains 1,303 instruction pairs generated via a human‑agent co‑creation framework in AI2‑THOR","Six leading embodied agents lose 19.65% task success and gain 54.10% empty‑plan rate under ambient context illusion","The phenomenon also appears in real‑world robots, raising safety concerns"],"whyItMatters":"It shows current embodied agents rely on unnecessary contextual language, limiting their robustness and safety when deployed in real‑world environments.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-11-04T05:00:00Z","updatedAt":"2026-11-04T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"VagueBench: Benchmarking the Robustness of Embodied Agents on Ambient Context Illusions","url":"https://arxiv.org/abs/2611.03765","publishedAt":"2026-11-04T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release VagueBench benchmark exposing ambient context illusion in agents\", 4 November 2026, https://digestai.news/story/researchers-release-vaguebench-benchmark-exposing-ambient-context-illu","publisher":"Digest AI","title":"Researchers release VagueBench benchmark exposing ambient context illusion in agents","datePublished":"2026-11-04T05:00:00Z","url":"https://digestai.news/story/researchers-release-vaguebench-benchmark-exposing-ambient-context-illu"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}