New GLARE model improves meeting continuation forecasting on MDFB benchmark
Researchers introduced the Meeting Dynamic Forecasting Benchmark (MDFB), built from 2,207 real‑world meetings and 24,794 forward‑looking queries. The benchmark tests a model’s ability to continue a meeting transcript given a partial dialogue and an active question, measuring both task utility and conversational realism without needing exact replication of the actual future.
Key points
- MDFB benchmark contains 2,207 real meetings and 24,794 future‑facing queries for multi‑turn continuation forecasting.
- GLARE uses adversarial imitation learning with a discriminator‑derived KL‑regularized reward to improve utility and human‑likeness.
- Human evaluation shows GLARE achieves 0.66 utility and 0.70 human‑likeness win rates, beating SFT and SPIN.
The paper presents GLARE, an adaptation of adversarial imitation learning for conditional language generation. A discriminator ranks the true continuation above samples from the current actor, and its score supplies a KL‑regularized policy reward that evolves as the actor improves. Human judges gave GLARE win rates of 0.66 for utility and 0.70 for human‑likeness, surpassing strong baselines such as supervised fine‑tuning (SFT) and SPIN, though still below genuine human continuations. MDFB also serves as a social‑reasoning arena for evaluating other general‑purpose and closed‑source models via reference‑assisted judgments.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- Competence-Gated Pooling Improves Hybrid Event Forecasts Over External Signals · 1 src
- Latent-Attention Masked Autoencoders Boost Multimodal Cardiac AI on 1.2M Hospital Stays · 1 src
- Language Models Fall Short for Quantitative Decision Tasks, Proposing Large Quantitative Models · 1 src
- GTA agent and GT Bench improve LLM graph reasoning accuracy · 2 src
- New Theory Links Memory Compression to Factual Hallucination in LLMs · 1 src
Comments
via GitHub Discussions