# Taste-Bench shows top AI agents barely exceed 60% accuracy

Digest AI · Research · published 2026-10-18T08:49:41Z

Canonical: https://digestai.news/story/taste-bench-shows-top-ai-agents-barely-exceed-60-accuracy

## Summary

Researchers from Microsoft and City University of Hong Kong introduced Taste-Bench, a 502‑question benchmark that evaluates AI agents' ability to make good long‑term decisions during multi‑step tasks. The test presents agents with a decision fork—two divergent paths from a shared trajectory—and asks a model to pick the better one, counting a question only if the answer is correct both ways.

Fourteen models from the Claude, GPT, Grok, DeepSeek, GLM, MiniMax and Mistral families were evaluated. The best performer, GPT‑5.6 Sol, averaged 59.7 percent, closely followed by GPT‑5.5 at 59.5 percent, while pure guessing would yield 25 percent. Accuracy rose to 62.3 percent when the decisive clue appeared early in the trace, but fell to 21 percent when the clue emerged only after extensive later work. A student model distilled from Qwen3.6‑27B improved its own fork‑selection accuracy from 30 percent to 47.9 percent, and when its judgments were added as advice to an agent, success on 41 held‑out SWE‑bench Pro tasks increased from 14.6 percent to 33.7 percent. The benchmark and code are publicly released.

## Key points

- Taste-Bench contains 502 binary-choice questions evaluating agents' long-term decision quality.
- GPT-5.6 Sol achieved the highest score at 59.7 percent, only slightly above GPT-5.5’s 59.5 percent.
- Teaching a student model (Qwen3.6‑27B) raised accuracy from 30 % to 47.9 % and doubled SWE‑bench Pro success to 33.7 %.

## Why it matters

The benchmark reveals that current AI agents struggle with foresight, limiting reliability for complex, multi‑step tasks, and shows that targeted training can markedly improve planning performance.

## Sources

1. [AI agents have bad taste in long-term decisions and a new benchmark proves it](https://the-decoder.com/ai-agents-have-bad-taste-in-long-term-decisions-and-a-new-benchmark-proves-it) (The Decoder, 2026-10-18)

Part of the developing story: [AI Agents Struggle With Accuracy And Language](https://digestai.news/thread/new-study-finds-ai-struggles-to-correct-false-claims-clearly-across-languages) (2 stories)

## Cite

Digest AI, "Taste-Bench shows top AI agents barely exceed 60% accuracy", 18 October 2026, https://digestai.news/story/taste-bench-shows-top-ai-agents-barely-exceed-60-accuracy

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/taste-bench-shows-top-ai-agents-barely-exceed-60-accuracy.json