{"version":1,"type":"story","url":"https://digestai.news/story/sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors","json":"https://digestai.news/story/sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors.json","markdown":"https://digestai.news/story/sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors.md","slug":"sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors","headline":"Sakana AI's Multi-Layered Review catches 73.43% of core-claim errors","summary":"Sakana AI released a research paper, *Beyond Imitation*, describing an AI‑assisted peer‑review system called Multi‑Layered Review (MLR). The system uses three off‑the‑shelf Claude agents – Haiku 3.5 for appendix summarization, Sonnet 4 for optional literature search, and Sonnet 4 for the main review – and runs on a three‑pass prompt chain inspired by Keshav’s Three‑Pass Approach. It processes up to ten pages of a paper’s main text without GPU or fine‑tuning, costing about $0.47 per review.\n\nThe authors built a Contradiction Benchmark by inserting 1,164 planted contradictions into 257 CC‑licensed papers from ACL, AISTATS, CVPR, ICML 2025 and NeurIPS 2024. MLR detected 73.43% of distance‑0 (core‑claim) errors with four reviews, roughly five times the 14.81% achieved by the best baseline. A single MLR review still caught 60.79%. On real retracted papers, performance fell to 16.11% exact matches. Correlation with human scores was moderate (Pearson 0.586 on ICLR 2025 submissions). The authors present MLR as a complementary tool rather than a replacement for human reviewers.","keyPoints":["MLR caught 73.43% of core-claim errors, about 5× the best baseline’s 14.81%.","System runs on off‑the‑shelf Claude Sonnet 4 and Haiku 3.5, costing about $0.47 per review.","Benchmark used 1,164 planted contradictions in 257 papers from ACL, AISTATS, CVPR, ICML 2025 and NeurIPS 2024."],"whyItMatters":"Demonstrates that AI reviewers can meaningfully detect substantive errors, moving beyond mimicry, and offers a low‑cost, off‑the‑shelf solution for researchers and publishers to improve paper quality.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Sakana AI"],"models":["Claude Sonnet 4","Claude Haiku 3.5","GPT-4.1","Gemini 2.5 Pro"],"people":[]},"firstPublishedAt":"2026-10-10T22:02:18Z","updatedAt":"2026-10-10T22:02:18Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"MarkTechPost","title":"Sakana AI’s LLM Peer Review System Catches 73% of Core-Claim Errors","url":"https://marktechpost.com/2026/10/10/sakana-ais-llm-peer-review-system-catches-73-of-core-claim-errors","publishedAt":"2026-10-10T22:02:18Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Sakana AI's Multi-Layered Review catches 73.43% of core-claim errors\", 10 October 2026, https://digestai.news/story/sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors","publisher":"Digest AI","title":"Sakana AI's Multi-Layered Review catches 73.43% of core-claim errors","datePublished":"2026-10-10T22:02:18Z","url":"https://digestai.news/story/sakana-ai-s-multi-layered-review-catches-73-43-of-core-claim-errors"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}