{"version":1,"type":"story","url":"https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf","json":"https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf.json","markdown":"https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf.md","slug":"nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf","headline":"Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format","summary":"The post on Reddit AI communities describes a new ternary 30B‑A3B checkpoint released by Nota AI, based on NVIDIA’s Nemotron 3.5 Lightning 30B‑A3B. The model runs on a 12 GB GPU with 172 tok/s decode and 7 000+ tok/s prefill, supporting 368 K context. It achieves roughly 89 % of the BF16 teacher’s quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36 and HRM8K 83.24 (average 83.82 vs 93.80). The checkpoint is a lossless re‑containerization of the ternary weights, compatible with llama.cpp and the PrismML fork, and is available for download.\n\nThe author notes that the model is fast, cheap, and well‑suited for long‑context reading, retrieval, summarisation and document QA, but it shows weaknesses in complex coding, mathematical reasoning and instruction‑following precision. The release demonstrates that ternary quantisation can preserve most of a large model’s performance while reducing memory and compute requirements.","keyPoints":["Nota AI releases ternary 30B‑A3B checkpoint of NVIDIA Nemotron 3.5 Lightning, runs on 12GB GPU with 172 tok/s decode","Model achieves 89% of BF16 teacher's quality, with GSM8K 89.31, MATH500 81.60, HumanEval 90.85, IFEval 83.36, HRM8K 83.24"],"whyItMatters":"The checkpoint shows that ternary quantisation can preserve most of a large model's performance while reducing memory and compute, enabling more efficient deployment on consumer GPUs.","category":{"slug":"models","name":"Generative AI & Models","url":"https://digestai.news/category/models"},"entities":{"companies":["Nota AI","NVIDIA CORPORATION & AFFILIATES"],"models":["Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF"],"people":[]},"firstPublishedAt":"2026-10-23T00:00:00Z","updatedAt":"2026-10-23T00:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"huggingface.co","title":"boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face","url":"https://huggingface.co/boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF","publishedAt":"2026-10-23T00:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1xfqi2d/boro64ternarynemotron35lightning30ba3bgguf/","points":null}],"thread":null,"cite":{"text":"Digest AI, \"Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format\", 23 October 2026, https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf","publisher":"Digest AI","title":"Nota AI releases ternary Nemotron 3.5 Lightning 30B-A3B model in GGUF format","datePublished":"2026-10-23T00:00:00Z","url":"https://digestai.news/story/nota-ai-releases-ternary-nemotron-3-5-lightning-30b-a3b-model-in-gguf"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}