Developing story · 2 episodes · 28 Sept to 3 Oct
Llama.cpp Optimizes Qwen4exp Vulkan Inference Performance
The saga tracks recent performance enhancements in the llama.cpp library, focusing on memory efficiency and kernel optimization for the Qwen4exp model. The latest development involves merging Vulkan kernel fusion for specific activation chains, following an earlier update that reduced indexer score memory usage.
-
Ggml-org halves indexer score memory in llama.cpp Pull Request
The ggml‑org team submitted a pull request to the llama.cpp repository that changes how the indexer scores are stored during inference. By giving each attention head its own product and summing the…
1 source primary source -
Ggml-org merges Vulkan kernel fusion for Qwen4exp's SCALE‑sigmoid‑SCALE chain
A pull request (29520) was merged into the ggml‑org/llama.cpp repository on Sep 28 2026, adding a fused Vulkan kernel that combines the SCALE‑SIGMOID‑SCALE‑hcpost operations used by the Qwen4exp…
1 source primary source
Who and what