# Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters

Digest AI · Agents & Tools · published 2026-09-18T05:19:31Z

Canonical: https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste

## Summary

An unofficial fork of llama.cpp introduces a persistent expert pool for mixture‑of‑experts (MoE) models that are offloaded with the --n-cpu-moe flag. Upstream llama.cpp lacks an expert cache, and earlier forks admitted zero tensors because a hard 3 GiB cap reserved the full context KV.

Reported gains on other hardware (e.g., +84 % on RTX 4090) were not reproduced. The test covers only 128 K context, one model, and one card, with no vision workloads or speculative decoding.

## Why it matters

Provides a usable expert‑cache for MoE inference on consumer‑grade AMD GPUs, potentially lowering PCIe traffic and latency for large‑context workloads, though benefits vary by use case.

## Sources

1. [llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters](https://github.com/atretador/gfx906-16gb-expert-pool) (github.com, 2026-09-18, primary source)

Part of the developing story: [Llama.cpp Integrates MTP Decoding for Qwen Models](https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50) (6 stories)

## Cite

Digest AI, "Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters", 18 September 2026, https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste.json