# SLATE benchmark tests long-horizon API selection in e-commerce automation

Digest AI · Research · published 2026-11-04T05:00:00Z

Canonical: https://digestai.news/story/slate-benchmark-tests-long-horizon-api-selection-in-e-commerce-automat

## Summary

Researchers introduced SLATE, a new benchmark designed to evaluate how well AI agents can automate complex workflows across thousands of APIs in e-commerce settings. Unlike existing tests, SLATE focuses on long-horizon, multi-step procedures—mapping queries to the correct tools without predefined step-by-step instructions—and uses a deterministic simulator to judge task success based on outcomes, not subjective scoring. The benchmark includes a library of roughly 1,000 tools, hierarchical task decomposition, and an automated metric that accepts valid but alternative approaches to completing tasks.

Experiments with six open decision models (ranging from 3B to 12B parameters) found that even the top-performing model achieved only a 5% success rate on SLATE’s tasks. The authors argue this demonstrates the benchmark’s difficulty and its potential to drive progress in reliable enterprise tool use. SLATE’s simulator, evaluation framework, and datasets are publicly available for further research.

## Key points

- SLATE benchmark tests AI agents on long-horizon API selection across ~1,000 e-commerce tools without predefined mappings
- Success is measured by task completion, not subjective LLM scoring, and accepts diverse valid trajectories
- Six open decision models (3B–12B params) achieved only 5% success, highlighting the benchmark’s challenge

## Why it matters

SLATE pushes AI tool-use research beyond toy tasks, forcing models to handle real-world complexity in enterprise automation. It could accelerate progress in reliable workflow automation, though current models struggle with its scale and variability.

## Sources

1. [SLATE: A Large-Scale Benchmark for Long-Horizon API Selection in E-commerce Tool Use](https://arxiv.org/abs/2611.03982) (arXiv cs.CL, 2026-11-04, primary source)

## Cite

Digest AI, "SLATE benchmark tests long-horizon API selection in e-commerce automation", 4 November 2026, https://digestai.news/story/slate-benchmark-tests-long-horizon-api-selection-in-e-commerce-automat

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/slate-benchmark-tests-long-horizon-api-selection-in-e-commerce-automat.json