{"version":1,"type":"story","url":"https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u","json":"https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u.json","markdown":"https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u.md","slug":"zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u","headline":"Zyphra releases FP8 FlashAttention kernel ZYFA for AMD MI355X, claims up to 1.76x speed boost","summary":"Zyphra released **ZYFA**, an open-source kernel library for AMD Instinct MI355X GPUs, on November 6. The FP8 and BF16 prefill attention kernels improve performance by **1.16–1.76x** over AMD’s default **AITER** library on real model activations. ZYFA’s FP8 error is **62–78% lower** than AITER’s at a head dimension of 128, according to Zyphra’s benchmarks, which include Mistral-7B, Qwen3-8B, and DeepSeek-V2-Lite models. The library runs on ROCm 7.2.3 and PyTorch 2.12, with a drop-in interface for FlashAttention functions. ZYFA’s best FP8 throughput reaches **2,672 TFLOP/s** at 32K tokens, though gains vary by model due to clock dependencies on operand bit flips. The project is part of Zyphra’s **Optimization Engine (ZOE)**, which powers its cloud service. Zyphra’s technical paper details optimizations like **Lean P** (replacing exponentials with multiply-adds) and a **two-wave ping-pong schedule** to maximize GPU utilization. The company also notes that FP8 attention on MI355X is **softmax-bound**, not matrix-bound, explaining why real-world gains differ from synthetic benchmarks. ZYFA’s accuracy remains within **a few percent** of other kernels like Composable Kernel and Triton, with FP64 comparisons showing **KL divergence** of **3.4×10⁻³** for Mistral-7B using ZYFA versus **5.9×10⁻³** with AITER. The library is **MI355X-only** and focuses on **prefill attention**, excluding decode or paged KV caching. Zyphra did not benchmark against NVIDIA GPUs and advises users to test sustained clocks on real activations for accurate results.","keyPoints":["ZYFA delivers **1.16–1.76x FP8 speedup** over AMD’s AITER on real model activations like Mistral-7B and Qwen3-8B","FP8 error drops **62–78% lower** than AITER at head dimension 128, with FP64 comparisons showing **3.4×10⁻³ KL divergence** for Mistral-7B","Library runs on **AMD MI355X GPUs only**, requires ROCm 7.2.3 and PyTorch 2.12, and is **open-source under Apache 2.0**"],"whyItMatters":"Faster FP8 kernels could reduce training/inference costs for long-context models on AMD hardware, though gains vary by model. ZYFA’s optimizations may influence future GPU kernel design, but its MI355X exclusivity limits immediate adoption outside AMD’s ecosystem.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":["Zyphra","AMD"],"models":[],"people":["Michal Sutter"]},"firstPublishedAt":"2026-11-06T17:28:04Z","updatedAt":"2026-11-06T17:28:04Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"MarkTechPost","title":"Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster","url":"https://marktechpost.com/2026/11/06/zyphra-zyfa-fp8-flashattention-amd-mi355x","publishedAt":"2026-11-06T17:28:04Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Zyphra releases FP8 FlashAttention kernel ZYFA for AMD MI355X, claims up to 1.76x speed boost\", 6 November 2026, https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u","publisher":"Digest AI","title":"Zyphra releases FP8 FlashAttention kernel ZYFA for AMD MI355X, claims up to 1.76x speed boost","datePublished":"2026-11-06T17:28:04Z","url":"https://digestai.news/story/zyphra-releases-fp8-flashattention-kernel-zyfa-for-amd-mi355x-claims-u"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}