Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ
The open-source repository ggml-org/llama.cpp merged a pull request (31188) on October 26, 2026, adding CUDA optimizations. The changes fuse MoE gate/up matrix multiplications and GLU (Gated Linear Unit) operations within the MMQ (Mixture of Matrix Queries) framework. The pull request was authored by anujj and approved by am17an and gaugarg-nv, both contributors to the project.
Key points
- Pull request #31188 merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQ
- Changes fuse operations to improve GPU inference performance for sparse or mixed-expert models
- Approved by contributors **am17an** and **gaugarg-nv**, merged on October 26, 2026
The update targets performance improvements for the ggml tensor library, which underpins machine learning workloads. The merged code replaces separate operations with fused CUDA kernels, potentially accelerating inference for models using sparse or mixed-expert architectures. No new benchmarks or performance figures were provided in the pull request, but the changes align with broader efforts to optimize GPU-based AI inference.
The story so far
4 episodes →- Ggml-org merges CUDA optimizations for MoE gate/up matmuls and GLU in MMQthis story
CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp
github.com · 28 October 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Hardware & Compute
All →- Core 4 Solutions lists 300-rack AI supercomputer with 9,600 Nvidia A100 GPUs on eBay for $59.7M · 2 src
- Nvidia and TSMC both hit records but stocks fall amid supply risks · 7 src
- Nvidia’s China AI chip shipments drop 44% as export limits and production constraints tighten · 4 src
- Nvidia proposes $2,000 annual fee per GPU for AI software · 8 src
- Oracle and Nvidia expand cloud AI infrastructure with Vera Rubin systems and new chip offerings · 7 src
Comments
via GitHub Discussions