# Ggml-org adds `--spec-draft-n-ubatch` to llama.cpp for batch control

Digest AI · Research · published 2026-10-14T11:08:21Z

Canonical: https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control

## Summary

`llama.cpp` now lets users set a custom physical batch size for draft context via the `--spec-draft-n-ubatch` flag. The change, merged in a pull request by **ggml-org**, allows drafts to process tokens in smaller chunks than the target model’s batch size. This reduces VRAM usage for the draft’s compute buffer while preserving the target’s KV cache efficiency. The default remains unchanged, matching the existing `--n-ubatch` behavior.

The update is part of ongoing optimizations for running large language models efficiently on limited hardware. It does not alter the draft’s final output but may improve performance for users working with constrained memory. The change was assisted by **Qwen3.8-Flash-Next** and documented in the repository’s test suite.

## Key points

- New `--spec-draft-n-ubatch` flag lets users set draft batch size independently of target model
- Smaller values shrink VRAM usage for draft compute buffer while preserving target KV cache
- Default behavior unchanged; flag is optional and requires no code changes

## Why it matters

This tweak helps developers run larger models on hardware with limited VRAM by decoupling draft batching from target model constraints.

## Sources

1. [common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp/pull/30459) (github.com, 2026-10-14, primary source)

Part of the developing story: [Llama.cpp Integrates MTP Decoding for Qwen Models](https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50) (4 stories)

## Cite

Digest AI, "Ggml-org adds `--spec-draft-n-ubatch` to llama.cpp for batch control", 14 October 2026, https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/ggml-org-adds-spec-draft-n-ubatch-to-llama-cpp-for-batch-control.json