Researchers release OncoNoteBERT for oncology note processing
A team of researchers introduced OncoNoteBERT, a specialized BERT-style encoder for natural language processing of real-world outpatient oncology notes. The model was trained on a governed UK dataset of 290,026 notes from 21,564 patients with lung and head-and-neck cancer. It outperformed existing models like RadBERT and PathologyBERT in perplexity scores (2.83 vs. 113.04 and 2035.03,…
Key points
- OncoNoteBERT achieves perplexity of **2.83** on oncology notes, outperforming RadBERT (**113.04**) and PathologyBERT (**2035.03**)
- Trained on **290,026 UK outpatient oncology notes** from **21,564 patients** with lung/head-and-neck cancer
- Model excels in tokenization efficiency and clinical probe predictions (**12/13**) vs. continued-pretraining variant (**7/13**)
The study highlights that continued pretraining and bespoke tokenization improve performance over generic biomedical models. Both OncoNote-RadBERT and OncoNoteBERT learned to represent institutional de-identification markers as single tokens, suggesting representation-layer design is critical for oncology NLP applications.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic study: task understanding beats job title for AI success · 1 src
- Researchers propose FEM-ASM to separate storage, execution, and coordination in language models · 1 src
- Researchers propose surrogate log-probabilities to audit LLM agents · 3 src
- Behavioral history outperforms descriptions for LLM synthetic personas · 1 src
- Researchers introduce ROAR to unify AI-driven research system runs · 1 src
Comments
via GitHub Discussions