Llama.cpp adds decision models for routing and scoring tasks
Llama.cpp now supports decision models that assign probabilities to predefined options in a single forward pass, unlike chat models that generate text. These models can route customer messages, moderate content, or guide agents’ next steps. The new /v1/systemone endpoint uses the System One format, compatible with existing clients. Models like Kev-4B-GGUF and OpenJev-GGUF (with vision support)…
Key points
- Decision models return probabilities for predefined options in one forward pass, unlike text-generating models
- Kev-4B-GGUF and OpenJev-GGUF (with image support) are available now via llama.cpp’s `/v1/systemone` endpoint
- Models can route customer queries, classify documents, or score urgency—useful for automation and agent workflows
Model page: Jev →
The story so far
5 episodes →- Llama.cpp adds decision models for routing and scoring tasksthis story
New in llama.cpp: Decision Models
huggingface.co · 2 October 2026
Loading the full article…
This text was published by huggingface.co. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- Gemini may delete existing text when asked to append, author reports · 1 src
- Google's Gemini outperforms ChatGPT and Claude as Android assistant · 1 src
- Meta launches Muse AI agent in Mexico, first in Latin America · 11 src
- Meta rolls out real-time voice and expressive avatars for Muse AI in the US · 7 src
- Meta improves Muse and Meta AI with Active Memory and Active Focus · 2 src
Comments
via GitHub Discussions