# vLLM 0.31 adds constrained decoding for 100% tool-call accuracy

Digest AI · Agents & Tools · published 2026-10-20T00:00:00Z

Canonical: https://digestai.news/story/vllm-0-31-adds-constrained-decoding-for-100-tool-call-accuracy

## Summary

Ubicloud found that open-weight models like Kimi K2.6 and DeepSeek V4 Flash often fail to generate correct tool calls, with vLLM version 0.28 achieving only 92% schema accuracy for Kimi K2.6. This led to nonsensical context and lower-quality results, especially in agentic workloads where multi-turn sessions are common. The issue stems from probabilistic token generation, where models may produce malformed calls due to mismatches between training data and production schemas.

However, vLLM disabled constrained decoding by default due to model-agnostic concerns, dropping Ubicloud’s accuracy to 91%. The company enabled the feature explicitly in vLLM 0.31 via the `--tool-strict-level` flag, achieving 100% schema accuracy for Kimi and DeepSeek. The technique uses XGrammar to restrict token sampling to valid paths, ensuring correct tool calls without significant performance overhead.

## Key points

- Constrained decoding in vLLM 0.31 guarantees 100% schema accuracy for Kimi and DeepSeek
- Ubicloud enabled the feature via `--tool-strict-level` flag after vLLM disabled it by default

## Why it matters

Tool-call accuracy directly impacts agentic AI reliability, reducing wasted retries and improving workflows for developers deploying open models.

## Sources

1. [Fixing Tool-Call Accuracy in vLLM](https://ubicloud.com/blog/fixing-tool-call-accuracy-in-vllm) (ubicloud.com, 2026-10-20)

Part of the developing story: [Intel's LLM Scaling Saga with vLLM Advances](https://digestai.news/thread/intel-releases-llm-scaler-vllm-pv-1-5) (2 stories)

## Cite

Digest AI, "vLLM 0.31 adds constrained decoding for 100% tool-call accuracy", 20 October 2026, https://digestai.news/story/vllm-0-31-adds-constrained-decoding-for-100-tool-call-accuracy

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/vllm-0-31-adds-constrained-decoding-for-100-tool-call-accuracy.json