{"version":1,"type":"story","url":"https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp","json":"https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp.json","markdown":"https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp.md","slug":"ggml-org-adds-fused-topk-mask-operation-to-llama-cpp","headline":"Ggml-org adds fused top‑k mask operation to llama.cpp","summary":"A pull request to the ggml‑org/llama.cpp repository introduces a new fused operation for QSA masking. The change adds the GGML_OP_TOP_K_MASK node to the CUDA backend, allowing the mask to be applied per‑head with optional score, block and extra tensors. The implementation builds on the existing top‑k mask logic but fuses the steps into a single kernel, reducing intermediate memory writes.\n\nThe fused op is referenced by several model families, including qwen4exp, glm5-next, DSA‑style attention models and the DeepSeek V4 CSA mask. The commit was authored by the user voidstackloop and co‑authored by Claude Opus 5.5 (anthropic.com). The update resolves a merge conflict with a prior change that added custom mailbox ops, keeping both pathways functional on backends that lack the new operation. Developers using llama.cpp on CUDA can now enable the op to improve performance of top‑k masking in attention layers.","keyPoints":["Adds GGML_OP_TOP_K_MASK fused kernel to ggml‑cuda for QSA masking","Supports per‑head top‑k selection with optional score, blocks, extra tensors","Integrated into llama.cpp and used by qwen4exp, glm5-next, DSA and DeepSeek V4 CSA models"],"whyItMatters":"The fused top‑k mask reduces memory traffic and speeds up attention masking on CUDA, giving llama.cpp users better performance for models that rely on QSA.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["ggml-org"],"models":["qwen4exp","glm5-next","DeepSeek V4 CSA"],"people":["Claude Opus 5.5","voidstackloop"]},"firstPublishedAt":"2026-10-16T00:00:00Z","updatedAt":"2026-10-16T00:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"github.com","title":"llama: create new fused op for QSA mask by am17an · Pull Request #30640 · ggml-org/llama.cpp","url":"https://github.com/ggml-org/llama.cpp/pull/30640","publishedAt":"2026-10-16T00:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1xbx12a/llama_create_new_fused_op_for_qsa_mask_by_am17an/","points":null}],"thread":{"title":"Llama.cpp Integrates MTP Decoding for Qwen Models","url":"https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50","storyCount":5},"cite":{"text":"Digest AI, \"Ggml-org adds fused top‑k mask operation to llama.cpp\", 16 October 2026, https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp","publisher":"Digest AI","title":"Ggml-org adds fused top‑k mask operation to llama.cpp","datePublished":"2026-10-16T00:00:00Z","url":"https://digestai.news/story/ggml-org-adds-fused-topk-mask-operation-to-llama-cpp"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}