Llama.cpp adds decision models for routing and scoring tasks
Llama.cpp now supports decision models that assign probabilities to predefined options in a single forward pass, unlike chat models that generate text. These models can route customer messages, moderate content, or guide agents’ next steps. The new /v1/systemone endpoint uses the System One format, compatible with existing clients. Models like Kev-4B-GGUF and OpenJev-GGUF (with vision support)…
Key points
- Decision models return probabilities for predefined options in one forward pass, unlike text-generating models
- Kev-4B-GGUF and OpenJev-GGUF (with image support) are available now via llama.cpp’s `/v1/systemone` endpoint
- Models can route customer queries, classify documents, or score urgency—useful for automation and agent workflows
Model page: Jev →
The story so far
7 episodes →- Llama.cpp adds decision models for routing and scoring tasksthis story
New in llama.cpp: Decision Models
huggingface.co · 2 October 2026
Loading the full article…
This text was published by huggingface.co. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- OpenAI research model uses crypto wallet funds without permission to finish task · 2 src
- Gmail Live rolls out to paid Workspace users · 2 src
- Researchers introduce ARIA framework for correcting multi-agent AI errors in real time · 1 src
- OpenAI introduces Dots agent for asynchronous task delegation · 4 src
- PrismML releases Ternary Bonsai 2 models for local AI on phones and wearables · 1 src
Comments
via GitHub Discussions