# Researchers introduce DischargeBench to evaluate LLMs as hospital discharge educators

Digest AI · Research · published 2026-09-21T04:00:00Z

Canonical: https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc

## Summary

A team of researchers presents DischargeBench, a persona‑grounded simulation that pits a candidate LLM educator against a Virtual Patient in multi‑turn discharge education dialogues. An Education Monitor Agent oversees patient realism without altering the educator, preserving the evaluation signal.

The benchmark draws on 477 cases extracted from MIMIC‑IV, covering 24 ICD chapters and varying along four persona axes: personality, education level, health literacy, and past‑medical‑history recall. Each simulated session is scored on Conversation Quality, Topic Checklist adherence, Comprehension, and Factual Consistency by an LLM‑as‑a‑Judge that is aligned with physician annotations.

Results across closed‑ and open‑source LLMs show that aggregate scores mask clinically relevant differences. Certain ICD chapters and difficult patient personas reveal coverage failures, comprehension gaps, and lower source‑answer agreement. The authors argue that LLM evaluation for discharge education should prioritize patient understanding rather than solely text quality or answer accuracy.

## Key points

- DischargeBench simulates multi‑turn discharge education dialogues between LLMs and a Virtual Patient.
- The benchmark includes 477 cases spanning 24 ICD chapters with persona dimensions like literacy and personality.
- Evaluation across LLMs shows performance varies by ICD chapter and patient persona, revealing comprehension gaps.

## Why it matters

A realistic, persona‑grounded benchmark lets developers measure how well LLMs ensure patient understanding after hospital stays, a critical step for safe AI use in healthcare.

## Sources

1. [From Discharge Notes to Patient Understanding: Persona-Grounded, Open-Ended Simulation of LLMs as Discharge Educators](https://arxiv.org/abs/2609.20827) (arXiv cs.CL, 2026-09-21, primary source)

## Cite

Digest AI, "Researchers introduce DischargeBench to evaluate LLMs as hospital discharge educators", 21 September 2026, https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-introduce-dischargebench-to-evaluate-llms-as-hospital-disc.json