qwenGPU compiles a GPU into Qwen3.5-2B weights
qwenGPU is an experimental model that embeds a 4‑lane SIMT GPU inside the weights of Qwen3.5‑2B. Developed by Vixhal and released on GitHub under MIT, it compiles CUDA to PTX and runs inside the model’s forward passes.
Key points
- qwenGPU adds 45 GPU layers to Qwen3.5‑2B, totaling 69 layers.
- Runs on Apple silicon, NVIDIA GPU or CPU with ~16 GB free memory, 10‑15 tps on Apple MPS.
- Reference interpreter matches Tesla T4 on 49/49 kernels, KL divergence ~7e‑4.
The model adds 45 untrained GPU layers to the 24 layers of Qwen3.5‑2B, for a total of 69 layers. It runs on Apple silicon, NVIDIA GPUs or CPU with about 16 GB free memory. Performance tests show exact launches, a KL divergence of about 7e‑4 from the base model, and a reference interpreter that matches an NVIDIA Tesla T4 on 49 of 49 kernels. Throughput is 10‑15 tokens per second on Apple MPS, indicating a research demo rather than production use.
qwenGPU uses Qwenputer’s transformer compiler and a Gated DeltaNet state to act as registers, ROM, and global memory. The demo includes a dot‑product kernel that returns 32 in 47 tokens. The project is still experimental, with no prebuilt checkpoint and limited reliability for a 2B model.
The story so far
5 episodes →- qwenGPU compiles a GPU into Qwen3.5-2B weightsthis story
Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA
MarkTechPost · 2 November 2026
Loading the full article…
This text was published by MarkTechPost and written by Michal Sutter. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- Anthropic renames $200 Claude plan to Ultra and removes 5-hour limit · 2 src
- JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP · Hugging Face · 1 src
- Google releases Gemini 4 Carbon for cyber defense · 1 src
- Amazon launches Alexa+ early access in Japan with generative AI for Prime members · 4 src
- Anthropic may have a new top-tier model in the works, report says · 1 src
Comments
via GitHub Discussions