Researchers propose MedCode to boost LLMs’ medical calculation accuracy by 20–30%
A new framework called MedCode aims to improve large language models’ ability to perform precise medical calculations. Current LLMs struggle with tasks requiring exact numerical outputs, such as medication dosing or organ-function assessments, where errors can have severe consequences. MedCode trains models to generate embedded, executable code for calculations, delegating arithmetic to a…
Key points
- MedCode framework embeds executable code in LLMs for medical calculations, improving accuracy by 20–30% on benchmarks
- Tests on LLaMA3-8B, Qwen2.5-7B, and Mistral-7B show gains in MedCalc and ICU scenario datasets
- Weighted Direct Preference Optimization (wDPO) adapts to emphasize difficult-to-distinguish medical calculation tasks
The approach uses supervised fine-tuning and preference datasets from the MedCalc benchmark and ICU scenarios. Researchers also introduce weighted Direct Preference Optimization (wDPO) to prioritize harder-to-distinguish preference pairs. Testing on LLaMA3-8B, Qwen2.5-7B, and Mistral-7B shows absolute accuracy gains of 20–30 percentage points, suggesting embedded code generation significantly improves reliability in medical calculations.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic's Claude begins 'dreaming' to reorganize memory in tests · 1 src
- John Langford quits Microsoft, declares AI Frontiers over · 1 src
- Anthropic releases study on Claude usage in scientific research · 3 src
- MIT and BASF release CheMeleon for low-data molecular prediction · 2 src
- Anthropic tests Claude agents in EVE Online trading and sabotage detection · 1 src
Comments
via GitHub Discussions