{"version":1,"type":"story","url":"https://digestai.news/story/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context","json":"https://digestai.news/story/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context.json","markdown":"https://digestai.news/story/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context.md","slug":"rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context","headline":"RBS-Attention provides radius-bounded sparse prefill for long-context LLMs","summary":"A new arXiv paper introduces RBS-Attention, a training‑free sparse‑prefill technique that uses a centroid base branch and a rescue branch to select attention blocks. The method independently thresholds each branch and merges their masks, preserving regular block‑sparse FlashAttention execution while mitigating the mean‑dilution failure mode.\n\nOn H100 GPUs, RBS-Attention delivers a 20.65× speedup over dense prefill‑attention, an 11.92× boost compared with vLLM prefill‑attention, and a 5.97× end‑to‑end time‑to‑first‑token improvement at a 128K context on Qwen3-30B-A3B-Instruct-2507-FP8. When applied to the dense Qwen3-32B model, overall RULER accuracy is 88.65 versus 89.52 for dense attention. Additional evaluations on LongBench‑v2, InfiniteBench, and Video‑MME confirm quality retention, and ablations explore block size, threshold, and memory behavior.","keyPoints":["RBS-Attention achieves 20.65× standalone prefill‑attention speedup on H100 GPUs.","End‑to‑end time‑to‑first‑token speedup is 5.97× at 128K context on Qwen3-30B-A3B-Instruct-2507-FP8.","Overall RULER accuracy drops to 88.65 versus 89.52 for dense attention."],"whyItMatters":"Improved sparse prefill cuts inference latency and compute for long‑context LLMs, making large‑prompt applications more practical.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Qwen3-30B-A3B-Instruct-2507-FP8","Qwen3-32B"],"people":[]},"firstPublishedAt":"2026-09-21T04:00:00Z","updatedAt":"2026-09-21T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"RBS-Attention: Radius-Bounded Sparse Prefill for Long-Context Large Language Models","url":"https://arxiv.org/abs/2609.20971","publishedAt":"2026-09-21T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"RBS-Attention provides radius-bounded sparse prefill for long-context LLMs\", 21 September 2026, https://digestai.news/story/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context","publisher":"Digest AI","title":"RBS-Attention provides radius-bounded sparse prefill for long-context LLMs","datePublished":"2026-09-21T04:00:00Z","url":"https://digestai.news/story/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}