{"version":1,"type":"story","url":"https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc","json":"https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc.json","markdown":"https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc.md","slug":"researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc","headline":"Researchers introduce DischargeBench to evaluate LLMs as hospital discharge educators","summary":"A team of researchers presents DischargeBench, a persona‑grounded simulation that pits a candidate LLM educator against a Virtual Patient in multi‑turn discharge education dialogues. An Education Monitor Agent oversees patient realism without altering the educator, preserving the evaluation signal.\n\nThe benchmark draws on 477 cases extracted from MIMIC‑IV, covering 24 ICD chapters and varying along four persona axes: personality, education level, health literacy, and past‑medical‑history recall. Each simulated session is scored on Conversation Quality, Topic Checklist adherence, Comprehension, and Factual Consistency by an LLM‑as‑a‑Judge that is aligned with physician annotations.\n\nResults across closed‑ and open‑source LLMs show that aggregate scores mask clinically relevant differences. Certain ICD chapters and difficult patient personas reveal coverage failures, comprehension gaps, and lower source‑answer agreement. The authors argue that LLM evaluation for discharge education should prioritize patient understanding rather than solely text quality or answer accuracy.","keyPoints":["DischargeBench simulates multi‑turn discharge education dialogues between LLMs and a Virtual Patient.","The benchmark includes 477 cases spanning 24 ICD chapters with persona dimensions like literacy and personality.","Evaluation across LLMs shows performance varies by ICD chapter and patient persona, revealing comprehension gaps."],"whyItMatters":"A realistic, persona‑grounded benchmark lets developers measure how well LLMs ensure patient understanding after hospital stays, a critical step for safe AI use in healthcare.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-09-21T04:00:00Z","updatedAt":"2026-09-21T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"From Discharge Notes to Patient Understanding: Persona-Grounded, Open-Ended Simulation of LLMs as Discharge Educators","url":"https://arxiv.org/abs/2609.20827","publishedAt":"2026-09-21T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers introduce DischargeBench to evaluate LLMs as hospital discharge educators\", 21 September 2026, https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc","publisher":"Digest AI","title":"Researchers introduce DischargeBench to evaluate LLMs as hospital discharge educators","datePublished":"2026-09-21T04:00:00Z","url":"https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}