{"version":1,"type":"story","url":"https://digestai.news/story/open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run","json":"https://digestai.news/story/open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run.json","markdown":"https://digestai.news/story/open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run.md","slug":"open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run","headline":"Open-source decision models Jeff-Qwen3.5-0.8B and Jeff-Gemma4-E2B run in ~30ms on local hardware","summary":"**Jeff** is a set of small, fast decision models fine-tuned from Qwen3.5 and Gemma 4, designed for zero-shot classification tasks. The 0.8B-parameter model runs in about **22ms** on an RTX PRO 6000 and **28ms** on an Apple M4 Max, returning calibrated probabilities for predefined options. It uses the same request format as **Jev** but is not affiliated with it. The project emphasizes speed, calibration, and local deployment—all training occurs on a single RTX PRO 6000 GPU without cloud dependency, using synthetic data generated by an open model (Qwen3.8-Flash-Next) on DGX Sparks.\n\nThe models excel at fast, well-calibrated choices between options—such as routing customer service calls or moderating content—without multi-step reasoning. Benchmark tests show they approach or surpass **Jev**’s performance on classification tasks but lag in reasoning-heavy scenarios like **BBH** or **JevBench**. Fine-tuning on domain-specific data (e.g., voice navigation) can drastically improve accuracy: a **31.7% to 95.8%** jump in held-out accuracy was achieved in under **30 minutes** on one GPU. The project releases weights under **Apache 2.0** and code under **MIT**, with training data sources listed separately.","keyPoints":["Jeff-Qwen3.5-0.8B and Jeff-Gemma4-E2B run in ~22ms (RTX 6000) and ~28ms (M4 Max) for zero-shot decisions","Fine-tuning on ~11k examples boosted accuracy from 31.7% to 95.8% in 30 minutes on one GPU","Models use local hardware only, with synthetic data generated by Qwen3.8-Flash-Next on DGX Sparks"],"whyItMatters":"These lightweight models enable fast, local decision-making for tasks like routing, moderation, or game AI without cloud costs or complex reasoning. Their speed and calibration make them practical for embedded systems or edge devices where latency matters.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["NVIDIA","TypeSafe"],"models":["Jev","Qwen3.5","Qwen3.8-Flash-Next","Gemma 4","AutoJev-27B","Jeff-Qwen3.5-0.8B"],"people":["Denis Yarats"]},"firstPublishedAt":"2026-09-28T20:23:36Z","updatedAt":"2026-09-28T20:23:36Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"github.com","title":"Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms","url":"https://github.com/firelex/jeff","publishedAt":"2026-09-28T20:23:36Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Hacker News","url":"https://news.ycombinator.com/item?id=49883844","points":59}],"thread":{"title":"Supersonic Labs Launches Local AI Decision Models","url":"https://digestai.news/thread/supersonic-labs-releases-julia-1-a-144-3m-parameter-open-decision-model","storyCount":2},"cite":{"text":"Digest AI, \"Open-source decision models Jeff-Qwen3.5-0.8B and Jeff-Gemma4-E2B run in ~30ms on local hardware\", 28 September 2026, https://digestai.news/story/open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run","publisher":"Digest AI","title":"Open-source decision models Jeff-Qwen3.5-0.8B and Jeff-Gemma4-E2B run in ~30ms on local hardware","datePublished":"2026-09-28T20:23:36Z","url":"https://digestai.news/story/open-source-decision-models-jeff-qwen3-5-0-8b-and-jeff-gemma4-e2b-run"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}