PyTorch 2.15 adds FP8 support and speeds up LLM decoding on Mac MPS
The PyTorch team at Hugging Face announced PyTorch 2.15, focusing on the MPS backend for Apple silicon. FP8 tensors (float8e4m3fn) are now supported on MPS, using half the memory of BF16 and enabling scaled matrix multiplication directly on the GPU. The release replaces many MPSGraph operations with dedicated Metal kernels, improving linear‑algebra routines, complex‑tensor support, and…
Key points
- MoE models decode 5–6.5× faster on M3 Ultra, e.g., Qwen3.6-35B-A3B up to 61.3 tokens/s
- FP8 tensors on MPS use half the memory of BF16 and support scaled matrix multiplication
- Video VAE decode memory drops from ~270 GiB to ~8 GiB after moving padding to Metal
Mixture‑of‑experts (MoE) models see a 5–6.5× decoding speed boost on an M3 Ultra: Qwen3.6‑35B‑A3B rises from 9.4 to 61.3 tokens/s and LFM2.5‑8B‑A1B from 23.6 to 123.6 tokens/s. Dense models also benefit from an 8× faster softmax (57 µs → 7 µs) and faster linear layers. Video VAE decoding memory drops from about 270 GiB to about 8 GiB after moving padding to Metal, and image‑editing quality improves, with PSNR climbing from 10.06 dB to 29.95 dB. PyTorch 2.15 requires macOS 15+ and Python 3.11+, and a preview of 2.16 promises full‑Metal attention and further MPSGraph migrations.
Model page: Qwen Image 2.1 →
MPS in PyTorch 2.15
Hugging Face · 20 October 2026
Loading the full article…
This text was published by Hugging Face and written by Irakli Salia. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Hardware & Compute
All →- NVIDIA’s Vera CPU delivers up to twice the throughput of x86 servers for Modal’s AI sandboxes · 9 src
- Yeo Kheng Meng releases xHCI98 driver adding USB4 and Thunderbolt support to Windows 98 SE · 1 src
- Nvidia reportedly receives $35 billion PRO 5500 orders from Alibaba, Tencent and ByteDance · 4 src
- Samsung expands memory chip fabs in Pyeongtaek amid AI boom · 1 src
- SpaceX and Nvidia plan to launch Starmind AI satellite in Q4 2027 · 1 src
Comments
via GitHub Discussions