{"version":1,"type":"story","url":"https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte","json":"https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte.json","markdown":"https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte.md","slug":"researchers-identify-context-poisoning-as-extreme-value-attention-inte","headline":"Researchers identify context poisoning as extreme-value attention interference in long-context language models","summary":"A new arXiv paper introduces the concept of context poisoning, describing how long-context language models struggle to use decisive evidence when irrelevant or confusing information is added to the prompt. The authors frame this as extreme-value interference in attention mechanisms, where the score for relevant information is bounded while distractor scores grow with their number. They derive a theoretical bound showing that maintaining accuracy requires the evidence margin to scale with the square root of the logarithm of distractor count. Experiments confirm that accuracy drops with more context when hard negatives are present, and that retrieve-then-reason architectures and contrastive training may help mitigate the issue.","keyPoints":["Context poisoning occurs when irrelevant context degrades a model's ability to find decisive evidence","The phenomenon is modeled as extreme-value interference in attention mechanisms","Accuracy drops in experiments with embedded hard negatives as context length increases"],"whyItMatters":"This research identifies a fundamental limitation in long-context AI systems, explaining why adding more information can hurt performance and guiding architectural improvements for reliable retrieval-augmented generation.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-09-22T04:00:00Z","updatedAt":"2026-09-22T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models","url":"https://arxiv.org/abs/2609.22101","publishedAt":"2026-09-22T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers identify context poisoning as extreme-value attention interference in long-context language models\", 22 September 2026, https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte","publisher":"Digest AI","title":"Researchers identify context poisoning as extreme-value attention interference in long-context language models","datePublished":"2026-09-22T04:00:00Z","url":"https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}