{"version":1,"type":"story","url":"https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste","json":"https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste.json","markdown":"https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste.md","slug":"llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste","headline":"Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters","summary":"An unofficial fork of llama.cpp introduces a persistent expert pool for mixture‑of‑experts (MoE) models that are offloaded with the --n-cpu-moe flag. Upstream llama.cpp lacks an expert cache, and earlier forks admitted zero tensors because a hard 3 GiB cap reserved the full context KV.\n\nReported gains on other hardware (e.g., +84 % on RTX 4090) were not reproduced. The test covers only 128 K context, one model, and one card, with no vision workloads or speculative decoding.","keyPoints":[],"whyItMatters":"Provides a usable expert‑cache for MoE inference on consumer‑grade AMD GPUs, potentially lowering PCIe traffic and latency for large‑context workloads, though benefits vary by use case.","category":{"slug":"agents","name":"Agents & Tools","url":"https://digestai.news/category/agents"},"entities":{"companies":["AtomicChat","ggml-org","memoriaru","mixa3607"],"models":["Qwen3.8-Flash-Next-GGUF, AD-3.84bpw-IQ4XS-M64","Qwen4exp"],"people":["byzhanghewei"]},"firstPublishedAt":"2026-09-18T05:19:31Z","updatedAt":"2026-09-18T05:19:31Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"github.com","title":"llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters","url":"https://github.com/atretador/gfx906-16gb-expert-pool","publishedAt":"2026-09-18T05:19:31Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1wjh7ox/llamacpp_expertpool_fork_for_qwen_38_flash_next/","points":null}],"thread":{"title":"Llama.cpp Integrates MTP Decoding for Qwen Models","url":"https://digestai.news/thread/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-tested-on-mi50","storyCount":6},"cite":{"text":"Digest AI, \"Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters\", 18 September 2026, https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste","publisher":"Digest AI","title":"Llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters","datePublished":"2026-09-18T05:19:31Z","url":"https://digestai.news/story/llama-cpp-expert-pool-fork-for-qwen-3-8-flash-next-iq4-16gb-vram-teste"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}