# Researchers fine-tune 406M model for meeting summaries with retrieved text spans

Digest AI · Research · published 2026-09-23T04:00:00Z

Canonical: https://digestai.news/story/researchers-fine-tune-406m-model-for-meeting-summaries-with-retrieved

## Summary

A new paper on arXiv introduces a method called *Retrieved-Span Training* to improve query-focused meeting summarization. The approach fine-tunes a 406M-parameter Fusion-in-Decoder model on retrieved text spans of up to 2,000 words instead of full transcripts. When tested on the QMSum dataset, the model achieves a **36.33 ROUGE-1** score, compared to **35.41** for a 1.2B-parameter baseline. The authors note the difference is not statistically significant, but the smaller model uses **one-third fewer parameters** and **less than half the peak inference memory** of the larger one.

The paper also shows that replacing the first 4,500 words of a transcript with 2,000 retrieved words improves performance by **1.55 ROUGE-1** on test data. Under a single prompt and scorer, the 406M model outperforms five proprietary hosted models by at least **6.2 ROUGE-1**, though the results lack human or factuality evaluation. The findings are limited to QMSum and automated metrics.

## Key points

- 406M-parameter model fine-tuned on 2,000-word retrieved text spans scores 36.33 ROUGE-1 on QMSum
- Smaller model uses one-third fewer parameters and half the memory of a 1.2B baseline
- Replacing first 4,500 transcript words with 2,000 retrieved words boosts performance by 1.55 ROUGE-1

## Why it matters

Efficient summarization of long meetings could reduce costs for transcription and retrieval systems, while smaller models lower deployment barriers for edge devices or low-resource settings.

## Sources

1. [Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum](https://arxiv.org/abs/2609.25028) (arXiv cs.CL, 2026-09-23, primary source)

## Cite

Digest AI, "Researchers fine-tune 406M model for meeting summaries with retrieved text spans", 23 September 2026, https://digestai.news/story/researchers-fine-tune-406m-model-for-meeting-summaries-with-retrieved

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-fine-tune-406m-model-for-meeting-summaries-with-retrieved.json