Researchers identify context poisoning as extreme-value attention interference in long-context language models
A new arXiv paper introduces the concept of context poisoning, describing how long-context language models struggle to use decisive evidence when irrelevant or confusing information is added to the prompt. The authors frame this as extreme-value interference in attention mechanisms, where the score for relevant information is bounded while distractor scores grow with their number. They derive a…
1 source primary source
Key points
- Context poisoning occurs when irrelevant context degrades a model's ability to find decisive evidence
- The phenomenon is modeled as extreme-value interference in attention mechanisms
- Accuracy drops in experiments with embedded hard negatives as context length increases
Read the original at arXiv cs.CL · by Meysam Ghaffari, Nina Fatehi, Bhaskar Sen, Nasim Sabetpour, Carlos Morato primary sourceOpen source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- Researchers propose zero-inference prospective term that boosts personal memory recall · 1 src
- PsyAgentBench tests if LLMs truly mimic human psychological biases · 1 src
- Indy Autonomous Challenge features first road-course overtakes, Unimore wins · 1 src
- Researchers unveil TangleDiff to design entangled protein hydrogels · 1 src
- Anthropic reports Claude leads 26% of its AI research and development tasks · 3 src
Comments
via GitHub Discussions