Delta Inference Engine runs DeepSeek-V4.1-Flash natively on Windows
The engine is a Windows port of Xolotl and Strata that uses consumer hardware, serving an OpenAI-compatible API at http://127.0.0.1:8080/v1 with a context window of up to 1 million tokens.
Key points
- Achieves 41 tokens per second output on an NVIDIA RTX 3090 GPU with 192 GB RAM.
- Requires 12 GB+ VRAM, 128 GB+ RAM, and 385 GB free NVMe SSD space.
According to benchmarks conducted on a system featuring an NVIDIA RTX 3090 GPU and 192 GB of RAM, the setup achieved output speeds of approximately 41 tokens per second. The system requires an NVIDIA card with at least 12 GB of VRAM, 128 GB or more of system RAM, and roughly 385 GB of free space on a high-speed PCIe NVMe SSD.
The project supports multiple quantization variants, including default ds4 Q2 files alongside formats from Unsloth and ISTA-DASLab. Setup is handled through automated scripts that check system requirements, install Python dependencies, and download checked model files directly from Hugging Face.
The story so far
3 episodes →- Delta Inference Engine runs DeepSeek-V4.1-Flash natively on Windowsthis story
DeepSeek-V4.1-Flash-1017 at 41 tok/s on one 3090
github.com · 28 October 2026Loading the full article…
This text was published by github.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- Meta launches Muse personal AI agent in Singapore · 1 src
- OpenAI adds PIN‑locked chats to ChatGPT, forgetting PIN erases them · 1 src
- Nvidia reportedly invests $3 billion in Meta's equity offering and Muse AI work · 39 src
- Mozilla adds AI form filling to Firefox Smart Window · 2 src
- Perplexity publishes guide to LLM gateway risks and mechanics · 1 src
Comments
via GitHub Discussions