# Ijah launches Metal‑based inference engine for Qwen 3.8‑Flash‑Next on Apple M5

Digest AI · Hardware & Compute · published 2026-10-19T17:22:07Z

Canonical: https://digestai.news/story/ijah-launches-metalbased-inference-engine-for-qwen-3-8flashnext-on-app

## Summary

The open‑source ijah engine runs on Apple M5 GPUs using raw Metal code and serves the Qwen3.8‑Flash‑Next model and its Swift‑1.5 fine‑tune via an OpenAI‑compatible HTTP API. It requires an Apple M5 Mac with Metal 4 tensor units, about 90 GB of free unified memory for a single 262K‑token context (≈81 GB of weights plus ≈28 KB per token), and roughly 185 GB of disk (83 GB pack + 102 GB n‑gram table). A second model adds about 4 GB when shared files are hard‑linked, and the optional DFlash2 drafter adds about 1.2 GB on disk and 1 GB of memory.

Performance tests on a 40‑core M5 Max (128 GB, macOS 27.0.1) show ijah’s quality is 4‑5× closer to the bf16 reference than oMLX and 1.7‑2× closer than Sushi. Memory peaks at 16 GB for ijah versus 78 GB for oMLX and 69 GB for Sushi. Agent sessions resume in ~0.2 s for later turns and ~0.3 s for a new session when prompts are reused.

## Key points

- ijah runs on Apple M5 GPUs with Metal, serving Qwen3.8‑Flash‑Next and Swift‑1.5‑Flash‑Next via OpenAI‑compatible API
- Full 262K‑token context needs ≈90 GB unified memory and ≈185 GB disk (83 GB model + 102 GB n‑gram table)

## Why it matters

Enables long‑context LLM inference on Apple silicon with high quality and modest memory, opening new on‑device AI possibilities for developers.

## Sources

1. [ijah --- M5+ Mac Engine for Qwen 3.8 Flash Next / Swift 1.5 Flash Next. Optimal Quality and Speed on 128gb](https://github.com/publicExcess/ijah) (github.com, 2026-10-19, primary source)

Part of the developing story: [Qwen Flash Next Hardware Optimization Race](https://digestai.news/thread/inco-ai-releases-dflash-2-draft-model-for-qwen3-8-flash-next) (2 stories)

## Cite

Digest AI, "Ijah launches Metal‑based inference engine for Qwen 3.8‑Flash‑Next on Apple M5", 19 October 2026, https://digestai.news/story/ijah-launches-metalbased-inference-engine-for-qwen-3-8flashnext-on-app

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/ijah-launches-metalbased-inference-engine-for-qwen-3-8flashnext-on-app.json