{"version":1,"type":"story","url":"https://digestai.news/story/sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli","json":"https://digestai.news/story/sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli.json","markdown":"https://digestai.news/story/sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli.md","slug":"sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli","headline":"SAGE system raises grant review agreement to kappa 0.58, beating baseline","summary":"Researchers introduced SAGE, a schema‑guided LLM that converts grant rubrics into structured checks and links each judgement to evidence in the application package. In an initial evaluation on 35 nonprofit grant applications, SAGE’s assessments were compared with 105 original competition reviews, yielding a fair ordinal agreement of kappa = 0.29.\n\nAfter the foundation inspected SAGE’s output, it conducted a criterion‑level re‑review that produced 202 assessments. In this assisted round, SAGE improved to kappa = 0.58 and outperformed a one‑prompt‑per‑criterion baseline, which recorded kappa = 0.33 on the same subset. The system also showed higher rank correlation and lower error rates. A claim‑level audit identified which parts of the structured draft were confirmed, disputed, or left unaddressed, demonstrating SAGE’s ability to generate detailed, evidence‑linked, and auditable drafts for expert correction.","keyPoints":["SAGE evaluated on 35 nonprofit grant applications, achieving kappa 0.29 in initial test.","In a re‑review of 202 assessments, SAGE reached kappa 0.58, surpassing the baseline kappa 0.33.","The system links rubric criteria to evidence, producing auditable drafts for expert correction."],"whyItMatters":"Improving consistency and transparency in grant reviews can help foundations allocate funds more fairly and efficiently.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-09-21T04:00:00Z","updatedAt":"2026-09-21T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"SAGE: Schema-Guided LLMs for Grant Review","url":"https://arxiv.org/abs/2609.20829","publishedAt":"2026-09-21T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"SAGE system raises grant review agreement to kappa 0.58, beating baseline\", 21 September 2026, https://digestai.news/story/sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli","publisher":"Digest AI","title":"SAGE system raises grant review agreement to kappa 0.58, beating baseline","datePublished":"2026-09-21T04:00:00Z","url":"https://digestai.news/story/sage-system-raises-grant-review-agreement-to-kappa-0-58-beating-baseli"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}