Ggml-org Optimizes Local AI Inference Performance
The saga tracks a series of technical updates to the llama.cpp library aimed at improving efficiency and compatibility for various AI models. The latest episode introduces Multi-Token Prediction support for Mellum2.1, following recent enhancements to memory usage and kernel acceleration.
-
Ggml-org adds FlintForCausalLM model to llama.cpp
The ggml‑org/llama.cpp repository merged pull request 30928, introducing the FlintForCausalLM model. The contribution was authored by user alexey‑rsi and approved by reviewers ggerganov and…
1 source primary source -
Ggml-org adds MTP support for Mellum2.1 in llama.cpp Pull Request
A pull request on the ggml-org/llama.cpp repository has been merged, adding MTP support for the Mellum2.1 model. The change was approved and integrated into the codebase. The article does not…
1 source primary source -
Ggml-org merges CUDA patch to accelerate low-bit MMVQ in llama.cpp
A pull request (30517) to the ggml‑org/llama.cpp repository adds a CUDA patch that addresses a regression on GCN and introduces a fast path for several low‑bit quantization types. The change returns…
1 source primary source -
Ggml-org halves indexer score memory in llama.cpp Pull Request
The ggml‑org team submitted a pull request to the llama.cpp repository that changes how the indexer scores are stored during inference. By giving each attention head its own product and summing the…
1 source primary source -
Ggml-org merges Vulkan kernel fusion for Qwen4exp's SCALE‑sigmoid‑SCALE chain
A pull request (29520) was merged into the ggml‑org/llama.cpp repository on Sep 28 2026, adding a fused Vulkan kernel that combines the SCALE‑SIGMOID‑SCALE‑hcpost operations used by the Qwen4exp…
1 source primary source