# Qwen models struggle with outdated facts in retrieved evidence

Digest AI · Research · published 2026-10-28T04:00:00Z

Canonical: https://digestai.news/story/qwen-models-struggle-with-outdated-facts-in-retrieved-evidence

## Summary

A new study on arXiv finds that AI agents often fail not because they miss relevant facts but because they cannot distinguish outdated information from correct updates. Researchers tested Qwen2.5-7B-Instruct and Qwen3-14B on the FactConsolidation task, where accuracy dropped sharply when evidence sets included competing, older statements—even if the newest fact was present.

With a 64-token evidence budget, Qwen2.5-7B-Instruct scored 45% accuracy, but performance plummeted to 32% when the budget expanded to 512 tokens. The same issue appeared in a 32K-history test, where BM25-64 outperformed BM25-512. The study suggests that retrieval alone cannot guarantee correct answers if agents cannot filter out stale information.

## Key points

- Qwen2.5-7B-Instruct accuracy drops from 45% to 32% when evidence budget grows from 64 to 512 tokens
- Obsolete facts hurt performance more than unrelated evidence of equal size for Qwen2.5 and Qwen3-14B
- Study uses FactConsolidation task from MemoryAgentBench to isolate retrieval vs. fact-staleness issues

## Why it matters

This research highlights a critical flaw in AI agents: retrieving correct facts doesn’t ensure correct answers if outdated versions remain in memory. It could drive better design for fact-checking and temporal reasoning in retrieval-augmented systems.

## Sources

1. [When Retrieval Is Not Enough: Obsolete Facts in Agent Memory](https://arxiv.org/abs/2610.36308) (arXiv cs.AI, 2026-10-28, primary source)

## Cite

Digest AI, "Qwen models struggle with outdated facts in retrieved evidence", 28 October 2026, https://digestai.news/story/qwen-models-struggle-with-outdated-facts-in-retrieved-evidence

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/qwen-models-struggle-with-outdated-facts-in-retrieved-evidence.json