# Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ

Digest AI · Hardware & Compute · published 2026-10-28T13:40:41Z

Canonical: https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in

## Summary

The open-source repository **ggml-org/llama.cpp** merged a pull request (#31188) on October 26, 2026, adding CUDA optimizations. The changes fuse **MoE gate/up matrix multiplications** and **GLU (Gated Linear Unit)** operations within the **MMQ (Mixture of Matrix Queries)** framework. The pull request was authored by **anujj** and approved by **am17an** and **gaugarg-nv**, both contributors to the project.

The update targets performance improvements for the **ggml tensor library**, which underpins machine learning workloads. The merged code replaces separate operations with fused CUDA kernels, potentially accelerating inference for models using sparse or mixed-expert architectures. No new benchmarks or performance figures were provided in the pull request, but the changes align with broader efforts to optimize GPU-based AI inference.

## Key points

- Pull request #31188 merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ
- Changes fuse operations to improve GPU inference performance for sparse or mixed-expert models
- Approved by contributors **am17an** and **gaugarg-nv**, merged on October 26, 2026

## Why it matters

Fused CUDA kernels for MoE and GLU operations could reduce latency and memory overhead for large-scale models, benefiting open-source AI inference stacks.

## Sources

1. [CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp/pull/31188) (github.com, 2026-10-28, primary source)

Part of the developing story: [Llama.cpp Expands Support For New AI Models](https://digestai.news/thread/add-nemotron-super-3-5-vl-and-nemotron-bolt-74b-a9b-support-by-frozenblade1224) (4 stories)

## Cite

Digest AI, "Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ", 28 October 2026, https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/ggml-org-merges-cuda-optimizations-for-moe-gate-up-matmuls-and-glu-in.json