{"version":1,"type":"story","url":"https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned","json":"https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned.json","markdown":"https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned.md","slug":"altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned","headline":"ALTK-Evolve tops τ²-bench retail leaderboard using GPT-4.1 with learned guidelines","summary":"Researchers from AgentToolkit posted results showing that a basic ReAct agent powered by GPT-4.1 reaches 67.5% Pass^4 on the τ²-bench retail domain, topping the October 12, 2026 leaderboard snapshot. The method, called ALTK-Evolve, extracts behavioral guidelines from the agent's own training-task attempts using the same GPT-4.1 model, then appends those guidelines to the system prompt for test tasks. No model weights are updated and no human annotation is used.\n\nOn held-out retail tasks, Mean Accuracy rises from 78.1% to 82.5% and Pass^4 from 57.5% to 67.5% with the full lossless guideline set (292 guidelines). On airline tasks, GPT-4.1 gains +7.5 accuracy points and +10 Pass^4 points. A compact high-support core (sup3, 14 guidelines) works best for airline reliability. The same approach also improves a GPT-5.5 agent on airline: accuracy +10 points to 82.5% and Pass^4 +15 points to 70.0%. Input token cost per conversation increases from ~$0.084 to ~$0.139 for retail, but conversations become slightly shorter and latency drops. The authors say prompt caching can reclaim most of the added cost.","keyPoints":["GPT-4.1 agent with learned guidelines scores 67.5% Pass^4 on τ²-bench retail, leading the Oct 12, 2026 leaderboard","Guidelines are extracted automatically from training runs using the same model, no weight updates or human labels","Method also lifts GPT-5.5 airline Pass^4 from 55% to 70% on held-out tasks"],"whyItMatters":"Prompt-level memory lets older models match or beat frontier models on agent benchmarks without fine-tuning, lowering the barrier for reliable customer-service agents.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["AgentToolkit","Sierra","OpenAI"],"models":["GPT-4.1","GPT-5.5","RAFT-30B-A3B","Claude Sonnet 4.5","GPT-5.2"],"people":[]},"firstPublishedAt":"2026-10-20T15:19:48Z","updatedAt":"2026-10-20T15:47:04Z","sourceCount":2,"hasPrimarySource":true,"sources":[{"outlet":"Hugging Face","title":"ALTK-Evolve: How learned memory tops the τ2-bench leaderboard on customer service tasks without a frontier model","url":"https://huggingface.co/blog/ibm-research/altk-evolve-bc","publishedAt":"2026-10-20T15:19:48Z","type":"primary","primary":true,"lead":true},{"outlet":"Unite.AI","title":"IBM Research Reports Learned Memory Tops τ2-Bench Service Benchmark","url":"https://unite.ai/ibm-research-reports-learned-memory-tops-tau-2-bench-service-benchmark","publishedAt":"2026-10-20T15:47:04Z","type":"press","primary":false,"lead":false}],"sourceNotes":null,"discussions":[],"thread":{"title":"AI Agents Struggle With Accuracy And Language","url":"https://digestai.news/thread/new-study-finds-ai-struggles-to-correct-false-claims-clearly-across-languages","storyCount":3},"cite":{"text":"Digest AI, \"ALTK-Evolve tops τ²-bench retail leaderboard using GPT-4.1 with learned guidelines\", 20 October 2026, https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned","publisher":"Digest AI","title":"ALTK-Evolve tops τ²-bench retail leaderboard using GPT-4.1 with learned guidelines","datePublished":"2026-10-20T15:19:48Z","url":"https://digestai.news/story/altk-evolve-tops-2-bench-retail-leaderboard-using-gpt-4-1-with-learned"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}