# Zyphra releases FP8 FlashAttention kernel ZYFA for AMD MI355X, claims up to 1.76x speed boost

Digest AI · Hardware & Compute · published 2026-11-06T17:28:04Z

Canonical: https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u

## Summary

Zyphra released **ZYFA**, an open-source kernel library for AMD Instinct MI355X GPUs, on November 6. The FP8 and BF16 prefill attention kernels improve performance by **1.16–1.76x** over AMD’s default **AITER** library on real model activations. ZYFA’s FP8 error is **62–78% lower** than AITER’s at a head dimension of 128, according to Zyphra’s benchmarks, which include Mistral-7B, Qwen3-8B, and DeepSeek-V2-Lite models. The library runs on ROCm 7.2.3 and PyTorch 2.12, with a drop-in interface for FlashAttention functions. ZYFA’s best FP8 throughput reaches **2,672 TFLOP/s** at 32K tokens, though gains vary by model due to clock dependencies on operand bit flips. The project is part of Zyphra’s **Optimization Engine (ZOE)**, which powers its cloud service. Zyphra’s technical paper details optimizations like **Lean P** (replacing exponentials with multiply-adds) and a **two-wave ping-pong schedule** to maximize GPU utilization. The company also notes that FP8 attention on MI355X is **softmax-bound**, not matrix-bound, explaining why real-world gains differ from synthetic benchmarks. ZYFA’s accuracy remains within **a few percent** of other kernels like Composable Kernel and Triton, with FP64 comparisons showing **KL divergence** of **3.4×10⁻³** for Mistral-7B using ZYFA versus **5.9×10⁻³** with AITER. The library is **MI355X-only** and focuses on **prefill attention**, excluding decode or paged KV caching. Zyphra did not benchmark against NVIDIA GPUs and advises users to test sustained clocks on real activations for accurate results.

## Key points

- ZYFA delivers **1.16–1.76x FP8 speedup** over AMD’s AITER on real model activations like Mistral-7B and Qwen3-8B
- FP8 error drops **62–78% lower** than AITER at head dimension 128, with FP64 comparisons showing **3.4×10⁻³ KL divergence** for Mistral-7B
- Library runs on **AMD MI355X GPUs only**, requires ROCm 7.2.3 and PyTorch 2.12, and is **open-source under Apache 2.0**

## Why it matters

Faster FP8 kernels could reduce training/inference costs for long-context models on AMD hardware, though gains vary by model. ZYFA’s optimizations may influence future GPU kernel design, but its MI355X exclusivity limits immediate adoption outside AMD’s ecosystem.

## Sources

1. [Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster](https://marktechpost.com/2026/11/06/zyphra-zyfa-fp8-flashattention-amd-mi355x) (MarkTechPost, 2026-11-06)

## Cite

Digest AI, "Zyphra releases FP8 FlashAttention kernel ZYFA for AMD MI355X, claims up to 1.76x speed boost", 6 November 2026, https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u.json