Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters
An unofficial fork of llama.cpp introduces a persistent expert pool for mixture‑of‑experts (MoE) models that are offloaded with the --n-cpu-moe flag. Upstream llama.cpp lacks an expert cache, and earlier forks admitted zero tensors because a hard 3 GiB cap reserved the full context KV.
Reported gains on other hardware (e.g., +84 % on RTX 4090) were not reproduced. The test covers only 128 K context, one model, and one card, with no vision workloads or speculative decoding.
The story so far
3 episodes →- Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parametersthis story
llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters
github.com · 18 September 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- OpenAI launches Dots agent for ChatGPT Pro and Business Premium · 4 src
- Claude Code builds a file-organizing AI agent in 30 minutes · 1 src
- Anthropic lets users mod Claude Code like video game plugins · 2 src
- Cresta builds Conductor agent builder on Claude Agent SDK · 1 src
- GitHub launches ReviewBench to test AI code review agents · 3 src
Comments
via GitHub Discussions