Researchers compare two fine-tuning methods for small medical AI models
A new paper on arXiv compares two ways to fine-tune small language models (≤9B parameters) for medical question answering: Uniform-across-Sequence (UaS) and Segment-weighted (SwS). UaS applies equal loss weighting to the full chain-of-thought (CoT) sequence, while SwS separately weights CoT and answer losses.
Key points
- SwS method improves general-purpose models **44.8%** on MedQA multiple-choice QA over zero-shot baseline
- UaS outperforms SwS for free-text QA tasks across nearly all tested models
- Medical-domain models show smaller gains or declines with SwS on multiple-choice QA
The study tests six general- and medical-domain models on multiple-choice (MedQA, EMS-MCQA) and free-text (Med-O1) tasks. SwS outperforms UaS on multiple-choice QA, with general-purpose models like Qwen3.5-4B seeing up to a 44.8% improvement over zero-shot baselines. However, medical-domain models show smaller or even negative gains. UaS performs best for free-text QA, with the 3B Llama-3.2 model nearly matching the performance of larger 7B medical-domain models. The authors also note that overly long CoT generation can hurt performance, suggesting task-specific CoT length limits may help.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Tencent releases WeVisOne-VL, a 2B/4B open vision model for detection, OCR and matting · 1 src
- Anthropic releases FlashProve open-source system for provable AI inference · 6 src
- Lama-dt outperforms baselines on meta-world and mujoco · 1 src
- Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context · 1 src
- SLATE benchmark tests long-horizon API selection in e-commerce automation · 1 src
Comments
via GitHub Discussions