# Zyphra releases ZHA, a sparse attention method for frozen models

Digest AI · Research · published 2026-11-03T01:09:49Z

Canonical: https://digestai.news/story/zyphra-releases-zha-a-sparse-attention-method-for-frozen-models

## Summary

Zyphra has released Zyphra Hierarchical Attention (ZHA), a sparse attention method designed to reduce the computational cost of long-context processing. Unlike existing sparse methods that still scan the entire context to find relevant tokens, ZHA uses a tree-structured indexer to search for keys hierarchically. This approach allows the model to keep the full KV cache while only reading the most relevant portions, addressing a bottleneck where indexing can consume 60 to 90% of layer time near 1M tokens.

The method is lightweight, adding only 81,920 trainable parameters to a frozen pretrained model, such as Qwen3.6-35B-A3B. Zyphra reports that ZHA can be distilled into these models using approximately 4B tokens of training data. In benchmarks, the deep-tree variant of ZHA achieved 63.4% on LongBench v2, outperforming dense attention's 58.0% while using only about 1.1% of keys. It also maintained high accuracy on RULER-13, staying within a few points of dense attention across reasoning and coding tasks.

However, performance varies by base model. On Nemotron-3.5-Lightning-30B-A3B, the deep-tree configuration dropped 25.3 points below dense attention on RULER-13. Speed improvements are currently theoretical; Zyphra has not released optimized kernels, and reported speedups of 373 to 417x at 1B tokens are based on modeled H100 BF16 roofline projections rather than measured benchmarks.

## Key points

- ZHA adds 81,920 trainable parameters to frozen models like Qwen3.6-35B-A3B.
- Deep-tree ZHA scores 63.4% on LongBench v2, beating dense attention's 58.0%.
- Speedups up to 417x at 1B tokens are modeled projections, not measured results.

## Why it matters

ZHA offers a potential path to efficient long-context inference by making the indexing process sparse. If the projected speedups hold in optimized implementations, it could significantly reduce the cost of processing very long documents, though current results rely on theoretical models.

## Sources

1. [Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention](https://marktechpost.com/2026/11/02/zyphra-hierarchical-attention-zha-sparse-attention) (MarkTechPost, 2026-11-03)

## Cite

Digest AI, "Zyphra releases ZHA, a sparse attention method for frozen models", 3 November 2026, https://digestai.news/story/zyphra-releases-zha-a-sparse-attention-method-for-frozen-models

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/zyphra-releases-zha-a-sparse-attention-method-for-frozen-models.json