{"version":1,"type":"story","url":"https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks","json":"https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks.json","markdown":"https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks.md","slug":"llama-cpp-adds-decision-models-for-routing-and-scoring-tasks","headline":"Llama.cpp adds decision models for routing and scoring tasks","summary":"Llama.cpp now supports decision models that assign probabilities to predefined options in a single forward pass, unlike chat models that generate text. These models can route customer messages, moderate content, or guide agents’ next steps. The new `/v1/systemone` endpoint uses the System One format, compatible with existing clients. Models like Kev-4B-GGUF and OpenJev-GGUF (with vision support) are available in the Decision models collection, with more expected weekly. Users can test models locally via `llama serve` and adjust confidence thresholds for automation decisions. The community-maintained Decision Index compares performance and speed across models.","keyPoints":["Decision models return probabilities for predefined options in one forward pass, unlike text-generating models","Kev-4B-GGUF and OpenJev-GGUF (with image support) are available now via llama.cpp’s `/v1/systemone` endpoint","Models can route customer queries, classify documents, or score urgency—useful for automation and agent workflows"],"whyItMatters":"Decision models cut latency and token costs for tasks needing deterministic choices, like routing or moderation, compared to text-based models.","category":{"slug":"agents","name":"Agents & Tools","url":"https://digestai.news/category/agents"},"entities":{"companies":["Llama.cpp","TypeSafe"],"models":["Kev-4B-GGUF","OpenJev-GGUF","Julia-1-GGUF","Jev"],"people":[]},"firstPublishedAt":"2026-10-02T14:25:06Z","updatedAt":"2026-10-02T14:25:06Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"huggingface.co","title":"New in llama.cpp: Decision Models","url":"https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp","publishedAt":"2026-10-02T14:25:06Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1wvv6im/new_in_llamacpp_decision_models/","points":null}],"thread":{"title":"The AI Model Format Evolution Saga","url":"https://digestai.news/thread/qwen3-8-27b-gguf-updated-with-new-tensor-layout","storyCount":5},"cite":{"text":"Digest AI, \"Llama.cpp adds decision models for routing and scoring tasks\", 2 October 2026, https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks","publisher":"Digest AI","title":"Llama.cpp adds decision models for routing and scoring tasks","datePublished":"2026-10-02T14:25:06Z","url":"https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}