DigestAI news desk

Cut through the AI noise.

Research

OpenAI's GPT-6.1-sol outperforms Jev on paused tasks, but lags in real-time

The paper introduces RealtimeBench, a benchmark of 180 tasks spanning six game types that continue running while an agent generates its response. By keeping the environment live, the authors can measure how timing affects performance of large‑language‑model agents.

1 source primary source

Key points

  • RealtimeBench evaluates agents on 180 continuously running tasks across six game types
  • GPT-6.1-sol beats Jev when games are paused, but Jev is faster when games keep running
  • TORSI training lifts Jev’s average score from 13.51 to 24.19 and helps three other models improve

Paired tests show that the larger GPT‑6.1‑sol model beats the faster Jev model when the games are paused, but Jev overtakes GPT‑6.1‑sol when the games keep running. To address this gap, the authors propose Two‑Timescale Offline Recursive Self‑Improvement (TORSI). A slow “teacher” analyses failures in paused games, writes reusable instructions, and a fast “student” applies them without further teacher calls during live play. After two TORSI rounds, Jev’s average test score rises from 13.51 to 24.19 on a 0‑100 scale, improving across all six game types, and the same instructions lift three other models without additional tuning.

These results suggest a path toward real‑time intelligence, where deliberation and action are coordinated across different timescales to keep agents effective in continuously changing environments.

Model pages: GPT-6.1 Sol → · Jev →

The story so far

4 episodes →
  1. OpenAI's GPT-6.1-sol outperforms Jev on paused tasks, but lags in real-timethis story
Read the original at arXiv cs.AI · by Ningxin Pan, Hanyu Li, Yeqing Shen, Ning Ding, Yehui Tang, Wentao Zhang primary sourceOpen source ↗
Topics · follow one to build your own front page

The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.

Comments

via GitHub Discussions

More in Research

All →

Related stories