{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t","json":"https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t.json","markdown":"https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t.md","slug":"researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t","headline":"Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context","summary":"Researchers introduced **Dory1.0-4B**, a 4-billion-parameter language model with a looped architecture that adjusts compute budgets dynamically. The model uses a normalized Transformer (nGPT) backbone with sliding-window and full-attention layers, split into input, recurrent, and output blocks. During inference, the recurrent block can loop multiple times, improving accuracy without increasing memory usage significantly.\n\nDory1.0-4B was pre-trained on **6 trillion tokens**, with the first **4.5 trillion** trained in NVFP4 precision. Post-training included supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR), using a difficulty-aware loop curriculum. The model outperforms larger models like **Qwen3.5-9B** and **Gemma 4 12B/26B-A4B** on coding, agentic, and long-context benchmarks. Looping also improves token efficiency, reducing output by **1.3–2.5x** on agentic tasks. The model’s checkpoints are open-sourced in BF16, MXFP8, NVFP4, and GGUF formats.","keyPoints":["Dory1.0-4B is a 4B-parameter looped model with 256K-token context, using a normalized Transformer architecture","Pre-trained on 6T tokens (4.5T in NVFP4), then fine-tuned with SFT and RLVR for looped inference","Outperforms larger models like Qwen3.5-9B and Gemma 4 on coding/agentic tasks, with 1.3–2.5x token efficiency gains"],"whyItMatters":"Dory1.0-4B demonstrates how smaller models with looped architectures can rival larger ones in performance while optimizing compute and memory. Its open-source release may accelerate research in efficient, scalable language models.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Dory1.0-4B","Qwen3.5-9B","Gemma 4 12B","Gemma 4 26B-A4B"],"people":[]},"firstPublishedAt":"2026-11-04T05:00:00Z","updatedAt":"2026-11-04T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"Dory1.0-4B: A Looped Normalized-GPT","url":"https://arxiv.org/abs/2611.03998","publishedAt":"2026-11-04T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context\", 4 November 2026, https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t","publisher":"Digest AI","title":"Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context","datePublished":"2026-11-04T05:00:00Z","url":"https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}