# sFNX runs Qwen3.8-Flash-Next at 382 tokens/s using three used V100 GPUs

Digest AI · Generative AI & Models · published 2026-10-26T21:11:25Z

Canonical: https://digestai.news/story/sfnx-runs-qwen3-8-flash-next-at-382-tokens-s-using-three-used-v100-gpu

## Summary

sFNX built an engine that runs the Qwen3.8-Flash-Next model at 382 tokens per second on a code‑edit benchmark. The setup uses three used Tesla V100‑32GB GPUs that cost about $1,800 in total and are connected via PCIe Gen3 without NVLink.

The engine splits every transformer layer across the three cards, letting each request read weights from all cards simultaneously. Six concurrent conversations are kept, each with the full 262,144‑token context, and the system also serves OpenAI‑style chat, Anthropic messages, tool calls and a picture front‑end. Quantizations UD‑Q4KXL and Q80 run in production, and the code relies on llama.cpp, CUDA 12.9 and custom NCCL‑like summation inside a CUDA graph.

Benchmarks show a range of runs, with the headline 382 tokens/s being the best of three greedy code‑edit runs. The configuration is documented in the repo’s docs, and the engine starts in about 80 seconds, exposing an API on port 8080.

## Key points

- sFNX runs Qwen3.8-Flash-Next at 382 tokens per second on three V100‑32GB GPUs costing ~$1,800.
- The engine splits every model layer across three cards, avoiding NVLink and using custom CUDA‑graph sums.
- Six concurrent 262,144‑token conversations are served, including OpenAI chat, Anthropic messages and picture front‑end.

## Why it matters

Shows that large‑scale LLMs can be deployed on inexpensive, older GPUs, lowering entry barriers for developers and small teams seeking high‑throughput inference.

## Sources

1. [I built an engine that runs Qwen-Flash-Next at nearly 400t/s on cheap(ish) hardware. 3x V100 32GB in an old R740. Originally forked from strata but the engine guts all built for the old Volta generation.](https://github.com/johnfink8/sfnx) (github.com, 2026-10-26, primary source)

Part of the developing story: [Qwen3.8 Flash Next Optimization Sprint](https://digestai.news/thread/inco-ai-releases-dflash-2-draft-model-for-qwen3-8-flash-next) (4 stories)

## Cite

Digest AI, "sFNX runs Qwen3.8-Flash-Next at 382 tokens/s using three used V100 GPUs", 26 October 2026, https://digestai.news/story/sfnx-runs-qwen3-8-flash-next-at-382-tokens-s-using-three-used-v100-gpu

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/sfnx-runs-qwen3-8-flash-next-at-382-tokens-s-using-three-used-v100-gpu.json