{"version":1,"type":"story","url":"https://digestai.news/story/arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou","json":"https://digestai.news/story/arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou.json","markdown":"https://digestai.news/story/arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou.md","slug":"arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou","headline":"arXiv study finds language models encode chemistry knowledge before output","summary":"A new paper on arXiv examines how five open-source language models process chemical functional groups. Researchers tested models on 92 molecules and 20 functional groups, finding that even without chemistry training, they organize functional groups by chemical family and encode chemical differences in consistent ways. For example, models place thiols (R–SH) near alcohols (R–OH) in their internal representations, showing structured knowledge before generating text.\n\nThe base model correctly named functional groups only 58.5% of the time, but its errors were mostly failures to respond rather than incorrect answers. Fine-tuning for chemistry shifted the models’ focus from prompt wording to functional-group structure. The study warns that relying solely on model outputs to evaluate chemical knowledge may underestimate the underlying information they encode.","keyPoints":["Five open-source models organize chemical functional groups by family in internal representations, even without chemistry training","Base model correctly identifies functional groups 58.5% of the time, but errors stem from silence, not wrong answers","Chemistry fine-tuning reorganizes model focus from prompt wording to functional-group structure"],"whyItMatters":"This research reveals that language models may hold deeper chemical knowledge than their outputs suggest, challenging how we evaluate AI agents for scientific tasks.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["language models"],"people":[]},"firstPublishedAt":"2026-11-04T05:00:00Z","updatedAt":"2026-11-04T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Functional-Group Knowledge Is Structured in Language Models Before It Is Spoken","url":"https://arxiv.org/abs/2611.03684","publishedAt":"2026-11-04T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"arXiv study finds language models encode chemistry knowledge before output\", 4 November 2026, https://digestai.news/story/arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou","publisher":"Digest AI","title":"arXiv study finds language models encode chemistry knowledge before output","datePublished":"2026-11-04T05:00:00Z","url":"https://digestai.news/story/arxiv-study-finds-language-models-encode-chemistry-knowledge-before-ou"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}