# Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context

Digest AI · Research · published 2026-11-04T05:00:00Z

Canonical: https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t

## Summary

Researchers introduced **Dory1.0-4B**, a 4-billion-parameter language model with a looped architecture that adjusts compute budgets dynamically. The model uses a normalized Transformer (nGPT) backbone with sliding-window and full-attention layers, split into input, recurrent, and output blocks. During inference, the recurrent block can loop multiple times, improving accuracy without increasing memory usage significantly.

Dory1.0-4B was pre-trained on **6 trillion tokens**, with the first **4.5 trillion** trained in NVFP4 precision. Post-training included supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR), using a difficulty-aware loop curriculum. The model outperforms larger models like **Qwen3.5-9B** and **Gemma 4 12B/26B-A4B** on coding, agentic, and long-context benchmarks. Looping also improves token efficiency, reducing output by **1.3–2.5x** on agentic tasks. The model’s checkpoints are open-sourced in BF16, MXFP8, NVFP4, and GGUF formats.

## Key points

- Dory1.0-4B is a 4B-parameter looped model with 256K-token context, using a normalized Transformer architecture
- Pre-trained on 6T tokens (4.5T in NVFP4), then fine-tuned with SFT and RLVR for looped inference
- Outperforms larger models like Qwen3.5-9B and Gemma 4 on coding/agentic tasks, with 1.3–2.5x token efficiency gains

## Why it matters

Dory1.0-4B demonstrates how smaller models with looped architectures can rival larger ones in performance while optimizing compute and memory. Its open-source release may accelerate research in efficient, scalable language models.

## Sources

1. [Dory1.0-4B: A Looped Normalized-GPT](https://arxiv.org/abs/2611.03998) (arXiv cs.CL, 2026-11-04, primary source)

## Cite

Digest AI, "Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context", 4 November 2026, https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-release-dory1-0-4b-a-4b-parameter-looped-model-with-256k-t.json