R2VC Boosts Fact‑Checking Accuracy by 13.74% on FEVER Using Modular Retrieval and Calibration
R2VC is a new modular framework that separates retrieval, reasoning, verification, and confidence calibration for automated fact‑checking. Built on an 8‑billion‑parameter backbone, it uses hybrid sparse‑plus‑dense Wikipedia search, a fine‑tuned generator that outputs structured verdicts, an NLI cross‑encoder for evidence‑based candidate selection, and a lightweight calibrator that estimates…
Key points
- R2VC improves FEVER accuracy by 13.74% using an 8B backbone and modular design.
- Candidate selection and confidence calibration are the biggest contributors to performance gains.
- Retrieval failures, especially wrong‑entity evidence, still dominate error cases.
On the FEVER benchmark, R2VC improves accuracy by 13.74 percentage points over a baseline system. Ablation studies show that removing the verifier‑based candidate selection drops accuracy to 76.24%, while omitting calibration nearly doubles the Brier score to 0.161, underscoring the importance of confidence estimation. A manual review of 250 errors indicates that retrieval failures—especially wrong‑entity evidence—remain the biggest obstacle.
These results demonstrate that a modular pipeline can significantly raise both predictive accuracy and the reliability of confidence estimates in open‑domain fact‑checking, offering a clearer diagnostic path for future improvements.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- Competence-Gated Pooling Improves Hybrid Event Forecasts Over External Signals · 1 src
- Latent-Attention Masked Autoencoders Boost Multimodal Cardiac AI on 1.2M Hospital Stays · 1 src
- Language Models Fall Short for Quantitative Decision Tasks, Proposing Large Quantitative Models · 1 src
- GTA agent and GT Bench improve LLM graph reasoning accuracy · 2 src
- New Theory Links Memory Compression to Factual Hallucination in LLMs · 1 src
Comments
via GitHub Discussions