Researchers test recursive knowledge building for science claim verification
A new study on arXiv explores how recursive knowledge construction affects agents’ ability to verify scientific claims. The authors tested two design choices: what evidence the agent stores and how it decides when to stop. They created a dataset called ProClaim-eval with 718 claims across molecular biology, nuclear physics, and astronomy, each labeled as supported or refuted by experts.
Key points
- ProClaim-eval dataset includes 718 expert-labeled claims from biology, physics, and astronomy
- Recursive evidence storage outperforms in-context accumulation for claim verification
- Trained judge matches ProClaim’s accuracy on three of five subsets, using Claude Sonnet 4.6
The team built a system called ProClaim that maintains an external evidence state, separate from the model’s context. Their findings show that structured evidence storage improves performance over in-context accumulation. A trained judge that reviews this evidence outperformed fixed-round stopping. When tested against other verification systems and general-purpose agents, ProClaim ranked in the top three, though a separately trained judge matched its accuracy on three of five subsets. The study suggests recursive, structured evidence handling could be key for scientific agents.
The story so far
2 episodes →- Researchers test recursive knowledge building for science claim verificationthis story
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- SemiAnalysis releases GDPx benchmark with Claude Opus 5.5 leading at 12% · 3 src
- Gemma-3-12b fails to detect misinformation when poisoned claims dominate retrieved evidence · 1 src
- Researchers release BILAL-BENCH to test Bengali AI models’ Islamic text fabrication · 1 src
- Researchers prune 729 attention heads to cut GPT-2-XL’s memorisation by 91.5% · 1 src
- arXiv paper finds shortcuts distort RAG benchmarking accuracy · 1 src
Comments
via GitHub Discussions