# Llama.cpp adds decision models for routing and scoring tasks

Digest AI · Agents & Tools · published 2026-10-02T14:25:06Z

Canonical: https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks

## Summary

Llama.cpp now supports decision models that assign probabilities to predefined options in a single forward pass, unlike chat models that generate text. These models can route customer messages, moderate content, or guide agents’ next steps. The new `/v1/systemone` endpoint uses the System One format, compatible with existing clients. Models like Kev-4B-GGUF and OpenJev-GGUF (with vision support) are available in the Decision models collection, with more expected weekly. Users can test models locally via `llama serve` and adjust confidence thresholds for automation decisions. The community-maintained Decision Index compares performance and speed across models.

## Key points

- Decision models return probabilities for predefined options in one forward pass, unlike text-generating models
- Kev-4B-GGUF and OpenJev-GGUF (with image support) are available now via llama.cpp’s `/v1/systemone` endpoint
- Models can route customer queries, classify documents, or score urgency—useful for automation and agent workflows

## Why it matters

Decision models cut latency and token costs for tasks needing deterministic choices, like routing or moderation, compared to text-based models.

## Sources

1. [New in llama.cpp: Decision Models](https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp) (huggingface.co, 2026-10-02, primary source)

Part of the developing story: [The AI Model Format Evolution Saga](https://digestai.news/thread/qwen3-8-27b-gguf-updated-with-new-tensor-layout) (5 stories)

## Cite

Digest AI, "Llama.cpp adds decision models for routing and scoring tasks", 2 October 2026, https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/llama-cpp-adds-decision-models-for-routing-and-scoring-tasks.json