Imperial College study finds LLMs cite unread pages, raising accuracy concerns
Researchers at Imperial College London examined why large language models frequently cite web pages without opening them, a behavior linked to outdated or misleading answers. Their study tested one proprietary model (OpenAI’s GPT-5.6 Luna) and six open-weight models against 225 facts updated since February 2026, including changes in FDA drug info, finance, sports, and SEC filings.
Key points
- Imperial College London study found LLMs cite unread web pages, increasing outdated answers by 6.4% and misleading citations by 6.4%
- GPT-5.6 Luna answered 53.6% of questions using only search snippets, not opening pages, per the study
- Providing correct Wikipedia pages directly reduced outdated answers for open-weight models like gemma-4-31b and gpt-oss-20b
The findings show models often rely on search snippets instead of full pages, leading to higher rates of incorrect answers and false citations. For example, GPT-5.6 Luna answered 53.6% of questions using only search snippets, while open-weight models varied between 2.6% and 76% in not opening pages. When models did open pages, outdated answers dropped from 6.4% to 1.3%, and misleading citations fell from 6.4% to 2.3%. The study also revealed that even when correct evidence was provided, models sometimes favored outdated information in their final answers.
AI Often Cites Web Pages It Has Never Even Read
Unite.AI · 16 October 2026
Loading the full article…
This text was published by Unite.AI and written by Martin Anderson. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- Microsoft AI chief says in-house model beats OpenAI's using quarter of compute · 2 src
- OpenMOSS releases MOSS-Image-SFT text-to-image model · 1 src
- Meta releases Muse Spark Open, an 816B-parameter open-weight multimodal model with 42B active parameters and 1M-token context · 24 src
- [AI News 10/16] Meta and Apple join forces. The era of running 816B massive AI on Macs is here. New announcements from NVIDIA, CERN, and Amazon|Haruki|AIを試す大学生 · 1 src
- Liquid AI releases d1-ultra decision model at $0.042 per million tokens · 1 src
Comments
via GitHub Discussions