Do We Need the Largest LLMs for Every Clinical Decision?
The authors present CERTA, a decision‑level statistical certificate that decides when a quantized model can replace its native BF16 reference for clinical verification. They evaluated eleven instruction‑tuned models, both general‑purpose and domain‑adapted, from three families (7-72B) on 2,500 clinical‑verification decisions drawn from de‑identified contexts of 2,484 patients, using…
Key points
- CERTA enables 8-bit quantized models to handle 99.8‑100% of clinical verification decisions across eleven models.
- 4-bit quantization cuts deployment cost to about 25.8% and 27.3% of the BF16 reference for Qwen2.5-72B and HuatuoGPT-o1-72B, with peak memory ~43 GB.
- Mean absolute F1 change on disease named‑entity benchmarks is 0.045, showing limited accuracy loss across models.
In the operational analysis, round‑to‑nearest 8‑bit quantization at BF16 storage served 99.8‑100% of held‑out decisions across all eleven models. A class‑conditional 4‑bit frontier reduced the amortized BF16‑equivalent deployment cost to 25.8% for Qwen2.5-72B and 27.3% for HuatuoGPT-o1-72B, with measured peak memory of 43.0 and 42.7 GB, below the nominal capacity of a single 48 GB accelerator (deployment on that hardware was not directly tested). Under the exact split‑sample certificate, costs were 26.7% and 27.9%. On external disease named‑entity benchmarks, the frontier‑selected NF4 configurations showed a mean absolute F1 change of 0.045, indicating limited accuracy loss. CERTA thus turns compression choices into a risk‑controlled deployment policy for this verification task.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- OpenAI and Google AI models discovered critical PHP flaw affecting millions of websites · 6 src
- OpenAI brings back Aleksander Madry to focus on AI safety · 2 src
- OpenAI claims AI-generated proof of quasi‑Riemann hypothesis · 2 src
- Nvidia's Astranomy study reaches 74.8% success in robot control with GPT-6 Astra · 1 src
- PHP patches critical RCE flaws discovered with GPT-5.6 Sol, affecting up to 70% of websites · 8 src
Comments
via GitHub Discussions