AdamW outperforms curvature-aware preconditioners in TabPFN v2.5 biomedical fine-tuning
The authors empirically evaluated five AdamW‑based preconditioning strategies while fine‑tuning TabPFN v2.5 on 59 biomedical tabular datasets, including Alzheimer’s disease, breast cancer, schizophrenia, significant memory concern, KEEL biomedical collections, and UCI benchmarks. They measured predictive accuracy, computational efficiency, and performed statistical significance testing. Their…
Key points
- AdamW optimizer achieved the highest predictive performance across 59 biomedical datasets.
- Five curvature-aware AdamW preconditioners showed no consistent improvement in accuracy or speed.
- Datasets spanned Alzheimer’s, breast cancer, schizophrenia, SMC, KEEL, and UCI biomedical benchmarks.
In contrast, the curvature‑aware preconditioners did not produce reliable gains in either accuracy or speed for any of the datasets examined. The study suggests that generic preconditioning methods may fail to capture the specific optimization dynamics of healthcare‑oriented tabular learning, highlighting a need for biomedical‑aware preconditioner designs tailored to foundation models like TabPFN.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Semantic Routing Calibration mitigates LLM over-refusal · 1 src
- AIBuildAI-2.5 ranks first on MLE-Bench with 73.3% medal rate · 1 src
- Researchers fine-tune 406M model for meeting summaries with retrieved text spans · 1 src
- Researchers test how language models handle numerical formats in word problems · 1 src
- Author pretrains language model End-to-End in Rust for $164 · 1 src
Comments
via GitHub Discussions