{"version":1,"type":"story","url":"https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in","json":"https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in.json","markdown":"https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in.md","slug":"ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in","headline":"Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ","summary":"The open-source repository **ggml-org/llama.cpp** merged a pull request (#31188) on October 26, 2026, adding CUDA optimizations. The changes fuse **MoE gate/up matrix multiplications** and **GLU (Gated Linear Unit)** operations within the **MMQ (Mixture of Matrix Queries)** framework. The pull request was authored by **anujj** and approved by **am17an** and **gaugarg-nv**, both contributors to the project.\n\nThe update targets performance improvements for the **ggml tensor library**, which underpins machine learning workloads. The merged code replaces separate operations with fused CUDA kernels, potentially accelerating inference for models using sparse or mixed-expert architectures. No new benchmarks or performance figures were provided in the pull request, but the changes align with broader efforts to optimize GPU-based AI inference.","keyPoints":["Pull request #31188 merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ","Changes fuse operations to improve GPU inference performance for sparse or mixed-expert models","Approved by contributors **am17an** and **gaugarg-nv**, merged on October 26, 2026"],"whyItMatters":"Fused CUDA kernels for MoE and GLU operations could reduce latency and memory overhead for large-scale models, benefiting open-source AI inference stacks.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":[],"models":[],"people":["anujj","am17an","gaugarg-nv"]},"firstPublishedAt":"2026-10-28T13:40:41Z","updatedAt":"2026-10-28T13:40:41Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"github.com","title":"CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp","url":"https://github.com/ggml-org/llama.cpp/pull/31188","publishedAt":"2026-10-28T13:40:41Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1xhodj3/cuda_fuse_moe_gateup_matmuls_and_glu_in_mmq_by/","points":null}],"thread":{"title":"Llama.cpp Expands Support For New AI Models","url":"https://digestai.news/thread/add-nemotron-super-3-5-vl-and-nemotron-bolt-74b-a9b-support-by-frozenblade1224","storyCount":4},"cite":{"text":"Digest AI, \"Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ\", 28 October 2026, https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in","publisher":"Digest AI","title":"Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ","datePublished":"2026-10-28T13:40:41Z","url":"https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}