{"version":1,"type":"story","url":"https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control","json":"https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control.json","markdown":"https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control.md","slug":"ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control","headline":"Ggml-org adds `--spec-draft-n-ubatch` to llama.cpp for batch control","summary":"`llama.cpp` now lets users set a custom physical batch size for draft context via the `--spec-draft-n-ubatch` flag. The change, merged in a pull request by **ggml-org**, allows drafts to process tokens in smaller chunks than the target model’s batch size. This reduces VRAM usage for the draft’s compute buffer while preserving the target’s KV cache efficiency. The default remains unchanged, matching the existing `--n-ubatch` behavior.\n\nThe update is part of ongoing optimizations for running large language models efficiently on limited hardware. It does not alter the draft’s final output but may improve performance for users working with constrained memory. The change was assisted by **Qwen3.8-Flash-Next** and documented in the repository’s test suite.","keyPoints":["New `--spec-draft-n-ubatch` flag lets users set draft batch size independently of target model","Smaller values shrink VRAM usage for draft compute buffer while preserving target KV cache","Default behavior unchanged; flag is optional and requires no code changes"],"whyItMatters":"This tweak helps developers run larger models on hardware with limited VRAM by decoupling draft batching from target model constraints.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["ggml-org"],"models":["Qwen3.8-Flash-Next"],"people":[]},"firstPublishedAt":"2026-10-14T11:08:21Z","updatedAt":"2026-10-14T11:08:21Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"github.com","title":"common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp","url":"https://github.com/ggml-org/llama.cpp/pull/30459","publishedAt":"2026-10-14T11:08:21Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1x5odh2/common_add_specdraftnubatch_by_ggerganov_pull/","points":null}],"thread":{"title":"Llama.cpp Integrates MTP Decoding for Qwen Models","url":"https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50","storyCount":4},"cite":{"text":"Digest AI, \"Ggml-org adds `--spec-draft-n-ubatch` to llama.cpp for batch control\", 14 October 2026, https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control","publisher":"Digest AI","title":"Ggml-org adds `--spec-draft-n-ubatch` to llama.cpp for batch control","datePublished":"2026-10-14T11:08:21Z","url":"https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}