# Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format

Digest AI · Generative AI & Models · published 2026-10-23T00:00:00Z

Canonical: https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf

## Summary

The post on Reddit AI communities describes a new ternary 30B‑A3B checkpoint released by Nota AI, based on NVIDIA’s Nemotron 3.5 Lightning 30B‑A3B. The model runs on a 12 GB GPU with 172 tok/s decode and 7 000+ tok/s prefill, supporting 368 K context. It achieves roughly 89 % of the BF16 teacher’s quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36 and HRM8K 83.24 (average 83.82 vs 93.80). The checkpoint is a lossless re‑containerization of the ternary weights, compatible with llama.cpp and the PrismML fork, and is available for download.

The author notes that the model is fast, cheap, and well‑suited for long‑context reading, retrieval, summarisation and document QA, but it shows weaknesses in complex coding, mathematical reasoning and instruction‑following precision. The release demonstrates that ternary quantisation can preserve most of a large model’s performance while reducing memory and compute requirements.

## Key points

- Nota AI releases ternary 30B‑A3B checkpoint of NVIDIA Nemotron 3.5 Lightning, runs on 12GB GPU with 172 tok/s decode
- Model achieves 89% of BF16 teacher's quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36, HRM8K 83.24

## Why it matters

The checkpoint shows that ternary quantisation can preserve most of a large model's performance while reducing memory and compute, enabling more efficient deployment on consumer GPUs.

## Sources

1. [boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face](https://huggingface.co/boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF) (huggingface.co, 2026-10-23, primary source)

## Cite

Digest AI, "Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format", 23 October 2026, https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf.json