{"version":1,"type":"story","url":"https://digestai.news/story/nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas","json":"https://digestai.news/story/nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas.json","markdown":"https://digestai.news/story/nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas.md","slug":"nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas","headline":"NVIDIA boosts MoE inference speed 42.5% on DGX Spark with Ling-3.0-flash","summary":"NVIDIA detailed three optimizations for Mixture-of-Experts (MoE) inference on its DGX Spark system, improving decode throughput by 42.5% for Ant Group’s inclusionAI Ling-3.0-flash model. The 124B-parameter model, with 5.1B active parameters per token, ran on a single DGX Spark (GB10, 128GB unified memory) using batch size one. The optimizations—Humming MXFP4 MoE kernels, an FP8 output layer, and built-in speculative decoding—reduced mean latency by 24.5% on 1,319 GSM8K questions while keeping accuracy flat at 96.89% to 96.82%.\n\nThe optimizations address memory bottlenecks: Humming kernels compile at runtime for specific shapes, cutting MoE computation time, while FP8 quantization halves reads of the LM head. Speculative decoding drafts three tokens and verifies four at once, though it requires careful integration to avoid silent fallbacks. NVIDIA warns that combining FP8 LM head and speculative decoding demands a shared LM-head module. Ling-3.0-flash, released under the MIT license, is designed for low per-token costs via sparse MoE and hybrid linear attention, with native 256K context support. Its benchmarks include 56.6 on SWE-Bench Pro and 93.2 on MathArena AIME 2026.","keyPoints":["NVIDIA’s optimizations boost Ling-3.0-flash GSM8K decode throughput 42.5% on DGX Spark","Humming kernels and FP8 LM head cut per-step costs; speculative decoding reduces step count","Model activates 5.1B of 124B parameters per token, runs on single DGX Spark with 128GB memory"],"whyItMatters":"Faster MoE inference on desktop systems like DGX Spark could lower barriers for developers running large models locally, reducing cloud costs and latency for single-user tasks.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":["NVIDIA","Ant Group inclusionAI","Moonshot AI","Artificial Analysis","OpenRouter"],"models":["Ling-3.0-flash","Ling-3.0-tiny","Ling 3.0"],"people":[]},"firstPublishedAt":"2026-11-03T20:28:22Z","updatedAt":"2026-11-03T20:28:22Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"MarkTechPost","title":"NVIDIA Shows 3 Ways to Speed Up MoE Inference on DGX Spark, Lifting Ling-3.0-flash Throughput 42.5%","url":"https://marktechpost.com/2026/11/03/nvidia-dgx-spark-moe-inference-ling-3-0-flash","publishedAt":"2026-11-03T20:28:22Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"Intel's LLM Scaling Saga with vLLM Advances","url":"https://digestai.news/thread/intel-releases-llm-scaler-vllm-pv-1-5","storyCount":3},"cite":{"text":"Digest AI, \"NVIDIA boosts MoE inference speed 42.5% on DGX Spark with Ling-3.0-flash\", 3 November 2026, https://digestai.news/story/nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas","publisher":"Digest AI","title":"NVIDIA boosts MoE inference speed 42.5% on DGX Spark with Ling-3.0-flash","datePublished":"2026-11-03T20:28:22Z","url":"https://digestai.news/story/nvidia-boosts-moe-inference-speed-42-5-on-dgx-spark-with-ling-3-0-flas"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}