{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening","json":"https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening.json","markdown":"https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening.md","slug":"researchers-release-benchmark-for-ai-in-systematic-review-screening","headline":"Researchers release benchmark for AI in systematic review screening","summary":"A new paper on arXiv introduces a benchmark dataset and framework for evaluating large language models in systematic review screening. The dataset contains labeled entries to test how well LLMs classify article relevance, addressing the imbalance between included and excluded articles. The authors propose SRBench, a tool for prompt experimentation and result analysis, alongside a use case demonstrating its application.\n\nThe work aims to improve evaluation methods for AI-assisted screening, which is often slow and labor-intensive. Existing metrics may not accurately reflect performance on imbalanced datasets, the authors argue. The paper does not specify the number of entries or studies but highlights the need for better tools to support AI-driven research workflows.","keyPoints":["New benchmark dataset and framework for AI-assisted systematic review screening published on arXiv","Tool called PromptSR supports prompt experimentation and result analysis for LLMs in screening tasks","Authors argue existing evaluation metrics may not account for class imbalance in screening datasets"],"whyItMatters":"Better benchmarking tools could accelerate AI adoption in research, reducing manual workloads in systematic reviews.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-09-28T04:00:00Z","updatedAt":"2026-09-28T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"A Benchmark Framework for Screening Automation in Systematic Reviews","url":"https://arxiv.org/abs/2609.30298","publishedAt":"2026-09-28T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release benchmark for AI in systematic review screening\", 28 September 2026, https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening","publisher":"Digest AI","title":"Researchers release benchmark for AI in systematic review screening","datePublished":"2026-09-28T04:00:00Z","url":"https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}