# Researchers release benchmark for AI in systematic review screening

Digest AI · Research · published 2026-09-28T04:00:00Z

Canonical: https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening

## Summary

A new paper on arXiv introduces a benchmark dataset and framework for evaluating large language models in systematic review screening. The dataset contains labeled entries to test how well LLMs classify article relevance, addressing the imbalance between included and excluded articles. The authors propose SRBench, a tool for prompt experimentation and result analysis, alongside a use case demonstrating its application.

The work aims to improve evaluation methods for AI-assisted screening, which is often slow and labor-intensive. Existing metrics may not accurately reflect performance on imbalanced datasets, the authors argue. The paper does not specify the number of entries or studies but highlights the need for better tools to support AI-driven research workflows.

## Key points

- New benchmark dataset and framework for AI-assisted systematic review screening published on arXiv
- Tool called PromptSR supports prompt experimentation and result analysis for LLMs in screening tasks
- Authors argue existing evaluation metrics may not account for class imbalance in screening datasets

## Why it matters

Better benchmarking tools could accelerate AI adoption in research, reducing manual workloads in systematic reviews.

## Sources

1. [A Benchmark Framework for Screening Automation in Systematic Reviews](https://arxiv.org/abs/2609.30298) (arXiv cs.CL, 2026-09-28, primary source)

## Cite

Digest AI, "Researchers release benchmark for AI in systematic review screening", 28 September 2026, https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-release-benchmark-for-ai-in-systematic-review-screening.json