Ggml-org adds fused top‑k mask operation to llama.cpp
A pull request to the ggml‑org/llama.cpp repository introduces a new fused operation for QSA masking. The change adds the GGMLOPTOPKMASK node to the CUDA backend, allowing the mask to be applied per‑head with optional score, block and extra tensors. The implementation builds on the existing top‑k mask logic but fuses the steps into a single kernel, reducing intermediate memory writes.
Key points
- Adds GGML_OP_TOP_K_MASK fused kernel to ggml‑cuda for QSA masking
- Supports per‑head top‑k selection with optional score, blocks, extra tensors
- Integrated into llama.cpp and used by qwen4exp, glm5-next, DSA and DeepSeek V4 CSA models
The fused op is referenced by several model families, including qwen4exp, glm5-next, DSA‑style attention models and the DeepSeek V4 CSA mask. The commit was authored by the user voidstackloop and co‑authored by Claude Opus 5.5 (anthropic.com). The update resolves a merge conflict with a prior change that added custom mailbox ops, keeping both pathways functional on backends that lack the new operation. Developers using llama.cpp on CUDA can now enable the op to improve performance of top‑k masking in attention layers.
Model page: Claude Opus 5.5 →
The story so far
5 episodes →- Ggml-org adds fused top‑k mask operation to llama.cppthis story
llama: create new fused op for QSA mask by am17an · Pull Request #30640 · ggml-org/llama.cpp
github.com · 16 October 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Nvidia's Astranomy study reaches 74.8% success in robot control with GPT-6 Astra · 1 src
- OpenAI's GPT-5.6 Sol and Google's Gemini 4 Argon find critical PHP flaw · 1 src
- PHP discloses PHP2Shell vulnerability affecting up to 70% of websites · 8 src
- Microsoft Research says Xinzhi Zhang and Zhao Song prove KLS conjecture with AI · 1 src
- OpenAI announces improved COBOL building and migration capabilities · 1 src
Comments
via GitHub Discussions