# PyTorch 2.15 adds FP8 support and speeds up LLM decoding on Mac MPS

Digest AI · Hardware & Compute · published 2026-10-20T00:00:00Z

Canonical: https://digestai.news/story/pytorch-2-15-adds-fp8-support-and-speeds-up-llm-decoding-on-mac-mps

## Summary

The PyTorch team at Hugging Face announced PyTorch 2.15, focusing on the MPS backend for Apple silicon.  FP8 tensors (float8e4m3fn) are now supported on MPS, using half the memory of BF16 and enabling scaled matrix multiplication directly on the GPU.  The release replaces many MPSGraph operations with dedicated Metal kernels, improving linear‑algebra routines, complex‑tensor support, and audio/video processing.

Mixture‑of‑experts (MoE) models see a 5–6.5× decoding speed boost on an M3 Ultra: Qwen3.6‑35B‑A3B rises from 9.4 to 61.3 tokens/s and LFM2.5‑8B‑A1B from 23.6 to 123.6 tokens/s.  Dense models also benefit from an 8× faster softmax (57 µs → 7 µs) and faster linear layers.  Video VAE decoding memory drops from about 270 GiB to about 8 GiB after moving padding to Metal, and image‑editing quality improves, with PSNR climbing from 10.06 dB to 29.95 dB.  PyTorch 2.15 requires macOS 15+ and Python 3.11+, and a preview of 2.16 promises full‑Metal attention and further MPSGraph migrations.

## Key points

- MoE models decode 5–6.5× faster on M3 Ultra, e.g., Qwen3.6-35B-A3B up to 61.3 tokens/s
- FP8 tensors on MPS use half the memory of BF16 and support scaled matrix multiplication
- Video VAE decode memory drops from ~270 GiB to ~8 GiB after moving padding to Metal

## Why it matters

The upgrades let developers run larger LLMs, video and audio models locally on Apple silicon, cutting cloud costs and expanding on‑device AI capabilities.

## Sources

1. [MPS in PyTorch 2.15](https://huggingface.co/blog/mps-pytorch-2-15) (Hugging Face, 2026-10-20, primary source)

## Cite

Digest AI, "PyTorch 2.15 adds FP8 support and speeds up LLM decoding on Mac MPS", 20 October 2026, https://digestai.news/story/pytorch-2-15-adds-fp8-support-and-speeds-up-llm-decoding-on-mac-mps

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/pytorch-2-15-adds-fp8-support-and-speeds-up-llm-decoding-on-mac-mps.json