# ALTK-Evolve tops τ²-bench retail leaderboard using GPT-4.1 with learned guidelines

Digest AI · Research · published 2026-10-20T15:19:48Z · updated 2026-10-20T15:47:04Z

Canonical: https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned

## Summary

Researchers from AgentToolkit posted results showing that a basic ReAct agent powered by GPT-4.1 reaches 67.5% Pass^4 on the τ²-bench retail domain, topping the October 12, 2026 leaderboard snapshot. The method, called ALTK-Evolve, extracts behavioral guidelines from the agent's own training-task attempts using the same GPT-4.1 model, then appends those guidelines to the system prompt for test tasks. No model weights are updated and no human annotation is used.

On held-out retail tasks, Mean Accuracy rises from 78.1% to 82.5% and Pass^4 from 57.5% to 67.5% with the full lossless guideline set (292 guidelines). On airline tasks, GPT-4.1 gains +7.5 accuracy points and +10 Pass^4 points. A compact high-support core (sup3, 14 guidelines) works best for airline reliability. The same approach also improves a GPT-5.5 agent on airline: accuracy +10 points to 82.5% and Pass^4 +15 points to 70.0%. Input token cost per conversation increases from ~$0.084 to ~$0.139 for retail, but conversations become slightly shorter and latency drops. The authors say prompt caching can reclaim most of the added cost.

## Key points

- GPT-4.1 agent with learned guidelines scores 67.5% Pass^4 on τ²-bench retail, leading the Oct 12, 2026 leaderboard
- Guidelines are extracted automatically from training runs using the same model, no weight updates or human labels
- Method also lifts GPT-5.5 airline Pass^4 from 55% to 70% on held-out tasks

## Why it matters

Prompt-level memory lets older models match or beat frontier models on agent benchmarks without fine-tuning, lowering the barrier for reliable customer-service agents.

## Sources

1. [ALTK-Evolve: How learned memory tops the τ2-bench leaderboard on customer service tasks without a frontier model](https://huggingface.co/blog/ibm-research/altk-evolve-bc) (Hugging Face, 2026-10-20, primary source)
2. [IBM Research Reports Learned Memory Tops τ2-Bench Service Benchmark](https://unite.ai/ibm-research-reports-learned-memory-tops-tau-2-bench-service-benchmark) (Unite.AI, 2026-10-20)

Part of the developing story: [AI Agents Struggle With Accuracy And Language](https://digestai.news/thread/new-study-finds-ai-struggles-to-correct-false-claims-clearly-across-languages) (3 stories)

## Cite

Digest AI, "ALTK-Evolve tops τ²-bench retail leaderboard using GPT-4.1 with learned guidelines", 20 October 2026, https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned.json