# Ggml-org adds fused top‑k mask operation to llama.cpp

Digest AI · Research · published 2026-10-16T00:00:00Z

Canonical: https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp

## Summary

A pull request to the ggml‑org/llama.cpp repository introduces a new fused operation for QSA masking. The change adds the GGML_OP_TOP_K_MASK node to the CUDA backend, allowing the mask to be applied per‑head with optional score, block and extra tensors. The implementation builds on the existing top‑k mask logic but fuses the steps into a single kernel, reducing intermediate memory writes.

The fused op is referenced by several model families, including qwen4exp, glm5-next, DSA‑style attention models and the DeepSeek V4 CSA mask. The commit was authored by the user voidstackloop and co‑authored by Claude Opus 5.5 (anthropic.com). The update resolves a merge conflict with a prior change that added custom mailbox ops, keeping both pathways functional on backends that lack the new operation. Developers using llama.cpp on CUDA can now enable the op to improve performance of top‑k masking in attention layers.

## Key points

- Adds GGML_OP_TOP_K_MASK fused kernel to ggml‑cuda for QSA masking
- Supports per‑head top‑k selection with optional score, blocks, extra tensors
- Integrated into llama.cpp and used by qwen4exp, glm5-next, DSA and DeepSeek V4 CSA models

## Why it matters

The fused top‑k mask reduces memory traffic and speeds up attention masking on CUDA, giving llama.cpp users better performance for models that rely on QSA.

## Sources

1. [llama: create new fused op for QSA mask by am17an · Pull Request #30640 · ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp/pull/30640) (github.com, 2026-10-16, primary source)

Part of the developing story: [Llama.cpp Integrates MTP Decoding for Qwen Models](https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50) (5 stories)

## Cite

Digest AI, "Ggml-org adds fused top‑k mask operation to llama.cpp", 16 October 2026, https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp.json