# Researchers identify context poisoning as extreme-value attention interference in long-context language models

Digest AI · Research · published 2026-09-22T04:00:00Z

Canonical: https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte

## Summary

A new arXiv paper introduces the concept of context poisoning, describing how long-context language models struggle to use decisive evidence when irrelevant or confusing information is added to the prompt. The authors frame this as extreme-value interference in attention mechanisms, where the score for relevant information is bounded while distractor scores grow with their number. They derive a theoretical bound showing that maintaining accuracy requires the evidence margin to scale with the square root of the logarithm of distractor count. Experiments confirm that accuracy drops with more context when hard negatives are present, and that retrieve-then-reason architectures and contrastive training may help mitigate the issue.

## Key points

- Context poisoning occurs when irrelevant context degrades a model's ability to find decisive evidence
- The phenomenon is modeled as extreme-value interference in attention mechanisms
- Accuracy drops in experiments with embedded hard negatives as context length increases

## Why it matters

This research identifies a fundamental limitation in long-context AI systems, explaining why adding more information can hurt performance and guiding architectural improvements for reliable retrieval-augmented generation.

## Sources

1. [Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models](https://arxiv.org/abs/2609.22101) (arXiv cs.CL, 2026-09-22, primary source)

## Cite

Digest AI, "Researchers identify context poisoning as extreme-value attention interference in long-context language models", 22 September 2026, https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-identify-context-poisoning-as-extreme-value-attention-inte.json