# AdamW outperforms curvature-aware preconditioners in TabPFN v2.5 biomedical fine-tuning

Digest AI · Research · published 2026-09-23T04:00:00Z

Canonical: https://digestai.news/story/adamw-outperforms-curvature-aware-preconditioners-in-tabpfn-v2-5-biome

## Summary

The authors empirically evaluated five AdamW‑based preconditioning strategies while fine‑tuning TabPFN v2.5 on 59 biomedical tabular datasets, including Alzheimer’s disease, breast cancer, schizophrenia, significant memory concern, KEEL biomedical collections, and UCI benchmarks. They measured predictive accuracy, computational efficiency, and performed statistical significance testing. Their results indicate that the standard AdamW optimizer consistently delivered the best overall performance and highest statistical ranking across the diverse tasks.

In contrast, the curvature‑aware preconditioners did not produce reliable gains in either accuracy or speed for any of the datasets examined. The study suggests that generic preconditioning methods may fail to capture the specific optimization dynamics of healthcare‑oriented tabular learning, highlighting a need for biomedical‑aware preconditioner designs tailored to foundation models like TabPFN.

## Key points

- AdamW optimizer achieved the highest predictive performance across 59 biomedical datasets.
- Five curvature-aware AdamW preconditioners showed no consistent improvement in accuracy or speed.
- Datasets spanned Alzheimer’s, breast cancer, schizophrenia, SMC, KEEL, and UCI biomedical benchmarks.

## Why it matters

The findings inform researchers that generic preconditioners may be inadequate for biomedical tabular models, prompting development of optimization techniques specifically tuned for healthcare data.

## Sources

1. [Do Existing Preconditioners Improve Biomedical Tabular Foundation Learning? An Empirical Study on TabPFN Optimization](https://arxiv.org/abs/2609.25013) (arXiv cs.AI, 2026-09-23, primary source)

## Cite

Digest AI, "AdamW outperforms curvature-aware preconditioners in TabPFN v2.5 biomedical fine-tuning", 23 September 2026, https://digestai.news/story/adamw-outperforms-curvature-aware-preconditioners-in-tabpfn-v2-5-biome

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/adamw-outperforms-curvature-aware-preconditioners-in-tabpfn-v2-5-biome.json