{"version":1,"type":"story","url":"https://digestai.news/story/blocklock-fixes-llm-inference-batching-for-reproducible-outputs","json":"https://digestai.news/story/blocklock-fixes-llm-inference-batching-for-reproducible-outputs.json","markdown":"https://digestai.news/story/blocklock-fixes-llm-inference-batching-for-reproducible-outputs.md","slug":"blocklock-fixes-llm-inference-batching-for-reproducible-outputs","headline":"BlockLock fixes LLM inference batching for reproducible outputs","summary":"BlockLock is a new method for ensuring that large language model (LLM) inference produces the same output for a given request, no matter what other requests are in its batch. Existing approaches either constrain every operator in the batch or isolate prefill stages, which limits shared compute or requires padding decode stages. BlockLock instead fixes the geometry of the protected request while allowing shared batched prefill and retaining full decode batching for other requests.\n\nThe authors propose two modes: BlockLock-Rows, which uses fixed-width rows, and BlockLock-Pack, which isolates a dummy segment and extracts the target request’s KV state for decode. Tests on five dense models—Qwen3-8B/14B, Llama-3.1-8B-Instruct, and Gemma-3-4B/12B-IT—across three GPUs (NVIDIA A6000, A100, H100) show both modes preserve target sequences and Top20-logit bits. BlockLock-Rows cuts latency by **53.5–55.0%** and boosts throughput by **115.2–122.2%** over vLLM’s batch-invariant method, without requiring custom kernel implementations.","keyPoints":["BlockLock fixes LLM inference geometry to guarantee reproducible outputs per request, even in batched deployments","Two modes tested: BlockLock-Rows (fixed-width rows) and BlockLock-Pack (terminal-anchored packing with dummy segment)","Latency drops **53.5–55.0%** and throughput rises **115.2–122.2%** vs. vLLM Batch-Invariant on five models"],"whyItMatters":"Reproducible LLM inference is critical for production systems where output consistency matters, like financial or legal workflows. BlockLock’s efficiency gains could reduce costs and latency in cloud deployments without sacrificing reliability.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Qwen3-8B","Qwen3-14B","Llama-3.1-8B-Instruct","Gemma-3-4B-IT","Gemma-3-12B-IT"],"people":[]},"firstPublishedAt":"2026-11-06T05:00:00Z","updatedAt":"2026-11-06T05:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"BlockLock: Reproducible LLM Inference with Fixed Batch Geometry","url":"https://arxiv.org/abs/2611.06977","publishedAt":"2026-11-06T05:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"BlockLock fixes LLM inference batching for reproducible outputs\", 6 November 2026, https://digestai.news/story/blocklock-fixes-llm-inference-batching-for-reproducible-outputs","publisher":"Digest AI","title":"BlockLock fixes LLM inference batching for reproducible outputs","datePublished":"2026-11-06T05:00:00Z","url":"https://digestai.news/story/blocklock-fixes-llm-inference-batching-for-reproducible-outputs"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}