{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens","json":"https://digestai.news/story/researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens.json","markdown":"https://digestai.news/story/researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens.md","slug":"researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens","headline":"Researchers release LAMB, a Latin language model trained on 74B tokens","summary":"It uses the ModernBERT architecture and was pretrained in two phases on over **74 billion Latin tokens**, with a novel masking method informed by morphology. The model was developed alongside two new Latin datasets: **Acervus**, containing **24 billion BERT WordPiece tokens** (including **760,000 samples** from damaged OCR transcriptions repaired by a ByT5 model), and **Corpus Trinum**, a dataset of over **100 million semantic triplets**—the first of its kind for Latin—enabling tasks like sentence similarity analysis.\n\nThe authors tested LAMB against **15 pretrained encoders** and **6 LAMB checkpoints**, evaluating it across **five tasks and 17 subtasks**. LAMB outperformed all competitors on **16 of 17 subtasks**, with a **0.8 percentage-point gap** on the remaining one. Their experiments also revealed that finetuning choices can matter more than additional pretraining data for certain tasks, and they analyzed how tokenizer behavior and syntactic structure influence performance.","keyPoints":["LAMB is a Latin language encoder pretrained on 74B tokens using ModernBERT architecture","Two new Latin datasets released: Acervus (24B tokens, 760K OCR-repaired samples) and Corpus Trinum (100M semantic triplets)","LAMB outperformed 15 other encoders on 16 of 17 subtasks in benchmark tests"],"whyItMatters":"LAMB advances NLP for classical languages by combining large-scale pretraining with domain-specific datasets, potentially improving tasks like translation, syntax analysis, and semantic search for Latin texts.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["LAMB","ModernBERT","ByT5"],"people":[]},"firstPublishedAt":"2026-10-26T04:00:00Z","updatedAt":"2026-10-26T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"LAMB: A Modern Contextual Language Model for Latin","url":"https://arxiv.org/abs/2610.31723","publishedAt":"2026-10-26T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release LAMB, a Latin language model trained on 74B tokens\", 26 October 2026, https://digestai.news/story/researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens","publisher":"Digest AI","title":"Researchers release LAMB, a Latin language model trained on 74B tokens","datePublished":"2026-10-26T04:00:00Z","url":"https://digestai.news/story/researchers-release-lamb-a-latin-language-model-trained-on-74b-tokens"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}