Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format
The post on Reddit AI communities describes a new ternary 30B‑A3B checkpoint released by Nota AI, based on NVIDIA’s Nemotron 3.5 Lightning 30B‑A3B. The model runs on a 12 GB GPU with 172 tok/s decode and 7 000+ tok/s prefill, supporting 368 K context. It achieves roughly 89 % of the BF16 teacher’s quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36 and HRM8K 83.24 (average…
Key points
- Nota AI releases ternary 30B‑A3B checkpoint of NVIDIA Nemotron 3.5 Lightning, runs on 12GB GPU with 172 tok/s decode
- Model achieves 89% of BF16 teacher's quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36, HRM8K 83.24
The author notes that the model is fast, cheap, and well‑suited for long‑context reading, retrieval, summarisation and document QA, but it shows weaknesses in complex coding, mathematical reasoning and instruction‑following precision. The release demonstrates that ternary quantisation can preserve most of a large model’s performance while reducing memory and compute requirements.
boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face
huggingface.co · 23 October 2026
Loading the full article…
This text was published by huggingface.co. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- Grok releases first seven minutes of its AI-generated Odyssey series · 1 src
- Daily AI news that isn't swayed by rumors. MetAI 'AI Tsushin' is now live · 1 src
- Hugging Face releases Carbon-X, a 14.8B open DNA language model · 1 src
- Am17an merges tensor conversion changes to Qwen in llama.cpp · 1 src
- Anthropic reports 3.6x jump in token volume after Claude Opus 5.5 launch · 1 src
Comments
via GitHub Discussions