NaiveAI releases Naive-N0.5-Flash 309B MoE model with 1M context under MIT license
NaiveAI released Naive-N0.5-Flash, an open-weight 309B mixture-of-experts model with 15.5B active parameters, built for coding and AI R&D. The model uses a hybrid attention stack of Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA) with GQA4, eliminating full-attention layers to support a native 1M-token context window. SWA uses a 128-token window while DSA selects…
Key points
- 309B MoE model with 15.5B active parameters, native 1M context via hybrid SWA-DSA attention
- Open weights and inference code under MIT license; API pricing at $0.10/$0.40/$0.01 per million tokens
- Trained on 3.25T tokens; runs on FP8 NVIDIA GPUs with ~315 GB weights; up to 2,000 tokens/s claimed
The model completed 3.25T tokens of multi-stage training: 50B tokens of Indexer Warmup, 3T tokens of Sparse Attention Training, and 200B tokens of Learning Rate Decay. NaiveAI reports inference speeds of 50 tokens/s per user in Standard mode and up to 2,000 tokens/s in Ultrafast mode via its NaiveRT inference system. Model weights occupy approximately 315 GB and require FP8-capable NVIDIA GPUs. Weights and inference code are released under the MIT license, with API pricing set at $0.10, $0.40, and $0.01 per million tokens for input, output, and cache reads respectively. The model builds on the open-weight MiMo-V2.5 base model from the Xiaomi MiMo team.
Naive-N0.5-Flash - 309B-A15.5B
huggingface.co · 27 September 2026
Loading the full article…
This text was published by huggingface.co. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- Apiiro finds 1,122 public GitHub files with secrets saved by Claude Code · 1 src
- Thinking Machines releases Inkling 2 Small, a 594B MoE model with 17B active parameters · 8 src
- OpenAI releases GPT-6 Cyber, a Critical-rated model that outperforms predecessors in vulnerability detection and alignment · 25 src
- Google releases Gemini Omni 1.2 Flash with 15-second video generation and 30% price cut to $0.07 per second · 10 src
- US accuses Chinese executive of allegedly sharing Nvidia chip info with Beijing · 11 src
Comments
via GitHub Discussions