LOLPlay introduces benchmark for real‑time visual agents in league of legends
LOLPlay benchmark includes 36 champion‑specific tasks and 82 subgoal checks in League of Legends.
Key points
- LOLPlay benchmark includes 36 champion‑specific tasks and 82 subgoal checks in League of Legends.
- GPT‑6 Astra tops the benchmark with 25.0 % task success at low/medium effort and 40.3 % subgoal success at medium effort.
- Higher reasoning effort does not improve performance, and longer requests reduce interaction rounds for Astra.
The authors evaluate three models across three reasoning‑effort settings on 324 task instances. GPT‑6 Astra achieves the highest task success rate of 25.0 % at low and medium effort, and the highest subgoal success rate of 40.3 % at medium effort. Higher reasoning effort does not improve performance for any model, and longer requests reduce interaction rounds for Astra.
Trajectory analysis shows incomplete subgoal composition and actions executed after scene changes, highlighting a need for agents that combine visual reasoning with timely action.
Model page: GPT-6 Astra →
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic releases FlashProve open-source system for provable AI inference · 7 src
- Gemini and Claude debate AI personality, ending in a three-way tie · 1 src
- Tencent releases WeVisOne-VL, a 2B/4B open vision model for detection, OCR and matting · 1 src
- Lama-dt outperforms baselines on meta-world and mujoco · 1 src
- Researchers release Dory1.0-4B, a 4B-parameter looped model with 256K-token context · 1 src
Comments
via GitHub Discussions