{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-raglog-for-fault-detection-in-rag-pipelines","json":"https://digestai.news/story/researchers-release-raglog-for-fault-detection-in-rag-pipelines.json","markdown":"https://digestai.news/story/researchers-release-raglog-for-fault-detection-in-rag-pipelines.md","slug":"researchers-release-raglog-for-fault-detection-in-rag-pipelines","headline":"Researchers release RAGLog for fault detection in RAG pipelines","summary":"Researchers introduced **RAGLog**, an open-source tool for detecting and localizing faults in **Retrieval-Augmented Generation (RAG)** pipelines without needing reference answers or code changes. The tool analyzes execution logs, converting them into a unified schema, then applies rule-based, embedding-based, and LLM-based checks to pinpoint faulty stages.\n\nRAGLog was tested on a dataset of **3,134 runs** from five RAG pipelines, achieving a pooled F1 score of **0.73** on naturally occurring faults using **Claude Haiku 5.5**. It outperformed **Claude Code (0.58)** and **Codex (0.52)** and matched or exceeded performance on injected faults. The tool processes runs in about **five seconds at $0.005 per run**, costing less and running faster than agent-based alternatives. The paper, posted on **arXiv**, also includes a live fault injector and a benchmark dataset with stage-level fault labels.","keyPoints":["RAGLog detects faults in RAG pipelines using execution logs, no reference answers or code changes","Achieves a pooled F1 score of 0.73 on naturally occurring faults with Claude Haiku 5.5","Processes runs in ~5 seconds at $0.005 each, cheaper and faster than agent baselines"],"whyItMatters":"RAGLog could reduce debugging time for teams using RAG systems, improving reliability in enterprise and research applications where pipeline failures go undetected.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Claude Haiku 5.5","Claude Code","Codex"],"people":[]},"firstPublishedAt":"2026-11-04T05:00:00Z","updatedAt":"2026-11-04T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"From Logs to Faults: Reference-Free Fault Localization for Retrieval-Augmented Generation Pipelines","url":"https://arxiv.org/abs/2611.03725","publishedAt":"2026-11-04T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release RAGLog for fault detection in RAG pipelines\", 4 November 2026, https://digestai.news/story/researchers-release-raglog-for-fault-detection-in-rag-pipelines","publisher":"Digest AI","title":"Researchers release RAGLog for fault detection in RAG pipelines","datePublished":"2026-11-04T05:00:00Z","url":"https://digestai.news/story/researchers-release-raglog-for-fault-detection-in-rag-pipelines"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}