Ijah launches Metal‑based inference engine for Qwen 3.8‑Flash‑Next on Apple M5
The open‑source ijah engine runs on Apple M5 GPUs using raw Metal code and serves the Qwen3.8‑Flash‑Next model and its Swift‑1.5 fine‑tune via an OpenAI‑compatible HTTP API. It requires an Apple M5 Mac with Metal 4 tensor units, about 90 GB of free unified memory for a single 262K‑token context (≈81 GB of weights plus ≈28 KB per token), and roughly 185 GB of disk (83 GB pack + 102 GB n‑gram…
Key points
- ijah runs on Apple M5 GPUs with Metal, serving Qwen3.8‑Flash‑Next and Swift‑1.5‑Flash‑Next via OpenAI‑compatible API
- Full 262K‑token context needs ≈90 GB unified memory and ≈185 GB disk (83 GB model + 102 GB n‑gram table)
Performance tests on a 40‑core M5 Max (128 GB, macOS 27.0.1) show ijah’s quality is 4‑5× closer to the bf16 reference than oMLX and 1.7‑2× closer than Sushi. Memory peaks at 16 GB for ijah versus 78 GB for oMLX and 69 GB for Sushi. Agent sessions resume in ~0.2 s for later turns and ~0.3 s for a new session when prompts are reused.
The story so far
2 episodes →- Ijah launches Metal‑based inference engine for Qwen 3.8‑Flash‑Next on Apple M5this story
ijah --- M5+ Mac Engine for Qwen 3.8 Flash Next / Swift 1.5 Flash Next. Optimal Quality and Speed on 128gb
github.com · 19 October 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Hardware & Compute
All →- Tenstorrent raises $1.5B Series E at $6.2B valuation · 3 src
- Meta brings Prometheus supercluster online in Ohio to train Muse models · 2 src
- Opinion: TSMC seen as AI chip stock with most upside through 2030 · 2 src
- DeepSeek plans to use Huawei Ascend 950DT for pre‑training by end of 2026 · 3 src
- Nvidia says Vera CPU delivers 3.4 TB/s bandwidth and higher single-thread performance · 6 src
Comments
via GitHub Discussions