# LHTB 2.0 benchmarks AI agents over three-hour tasks with 77 new scenarios

Digest AI · Research · published 2026-10-14T04:00:00Z

Canonical: https://digestai.news/story/lhtb-2-0-benchmarks-ai-agents-over-three-hour-tasks-with-77-new-scenar

## Summary

Researchers released Long-Horizon-Terminal-Bench 2.0 (LHTB 2.0) on arXiv, expanding a test suite for AI agents that measure sustained progress over long workflows. The update adds 31 extended tasks—bringing the total to 77 across 13 domains—while fixing defects in the original 46. Tasks are generated from real-world artifacts using a parameterized pipeline, with three held out for evaluation and six used as behavioral evidence. The evaluation window now spans three hours, with checkpoints every 30 minutes to track progress dynamically rather than just final outcomes.

LHTB 2.0 scores 1,145 runs from 15 frontier models. GPT-6-astra leads with a dense mean reward of 0.69 and succeeds at 37 tasks with a score ≥0.95. GPT-5.6-sol follows with 0.67 and 32 tasks at the same threshold. Binary verifier scores average 0.221 on reconstruction tasks, while dense ladder scores average 0.653, highlighting how outcome-only metrics miss intermediate progress.

## Key points

- LHTB 2.0 adds 31 extended tasks to its original 46, totaling 77 across 13 domains with three-hour evaluation windows
- GPT-6-astra scores highest with a dense mean reward of 0.69 and succeeds at 37 tasks with ≥0.95 threshold
- Dense ladder scores average 0.653, showing binary verifiers undercount progress by averaging 0.221 on reconstruction tasks

## Why it matters

LHTB 2.0 shifts focus from binary pass/fail to measuring real-world agent progress over hours, revealing gaps in current evaluation methods.

## Sources

1. [Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress](https://arxiv.org/abs/2610.14146) (arXiv cs.AI, 2026-10-14, primary source)

## Cite

Digest AI, "LHTB 2.0 benchmarks AI agents over three-hour tasks with 77 new scenarios", 14 October 2026, https://digestai.news/story/lhtb-2-0-benchmarks-ai-agents-over-three-hour-tasks-with-77-new-scenar

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/lhtb-2-0-benchmarks-ai-agents-over-three-hour-tasks-with-77-new-scenar.json