RBS-Attention provides radius-bounded sparse prefill for long-context LLMs
A new arXiv paper introduces RBS-Attention, a training‑free sparse‑prefill technique that uses a centroid base branch and a rescue branch to select attention blocks. The method independently thresholds each branch and merges their masks, preserving regular block‑sparse FlashAttention execution while mitigating the mean‑dilution failure mode.
Key points
- RBS-Attention achieves 20.65× standalone prefill‑attention speedup on H100 GPUs.
- End‑to‑end time‑to‑first‑token speedup is 5.97× at 128K context on Qwen3-30B-A3B-Instruct-2507-FP8.
- Overall RULER accuracy drops to 88.65 versus 89.52 for dense attention.
On H100 GPUs, RBS-Attention delivers a 20.65× speedup over dense prefill‑attention, an 11.92× boost compared with vLLM prefill‑attention, and a 5.97× end‑to‑end time‑to‑first‑token improvement at a 128K context on Qwen3-30B-A3B-Instruct-2507-FP8. When applied to the dense Qwen3-32B model, overall RULER accuracy is 88.65 versus 89.52 for dense attention. Additional evaluations on LongBench‑v2, InfiniteBench, and Video‑MME confirm quality retention, and ablations explore block size, threshold, and memory behavior.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- TatBLiMP benchmarks Tatar linguistic minimal pairs · 1 src
- Recursive language models generalize out of domain, study shows · 1 src
- reviser proposes cursor-based text generation · 1 src
- SAGE system raises grant review agreement to kappa 0.58, beating baseline · 1 src
- Qwen2.5-Omni-3B adapters boost entity recall in accented conversational ASR · 1 src
Comments
via GitHub Discussions