Llama.cpp adds decision models for routing and scoring tasks
Llama.cpp now supports decision models that assign probabilities to predefined options in a single forward pass, unlike chat models that generate text. These models can route customer messages, moderate content, or guide agents’ next steps. The new /v1/systemone endpoint uses the System One format, compatible with existing clients. Models like Kev-4B-GGUF and OpenJev-GGUF (with vision support)…
Key points
- Decision models return probabilities for predefined options in one forward pass, unlike text-generating models
- Kev-4B-GGUF and OpenJev-GGUF (with image support) are available now via llama.cpp’s `/v1/systemone` endpoint
- Models can route customer queries, classify documents, or score urgency—useful for automation and agent workflows
Model page: Jev →
The story so far
5 episodes →- Llama.cpp adds decision models for routing and scoring tasksthis story
New in llama.cpp: Decision Models
huggingface.co · 2 October 2026
Loading the full article…
This text was published by huggingface.co. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- AWS shows multi-turn RL fine-tuning for search agents on SageMaker AI · 1 src
- OpenAI launches Dots, always-on AI agents in ChatGPT · 63 src
- Anthropic introduces mods for Claude Code · 1 src
- Microsoft adds Copilot Home, Code, and Autopilot to automate work tasks · 2 src
- Rhun, an open-source assembly code editor, adds optional AI features · 1 src
Comments
via GitHub Discussions