{"version":1,"type":"story","url":"https://digestai.news/story/artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag","json":"https://digestai.news/story/artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag.json","markdown":"https://digestai.news/story/artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag.md","slug":"artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag","headline":"Artificial Analysis releases open-source tool to benchmark local AI agents on laptops and workstations","summary":"Artificial Analysis has launched **AA-AgentPerf-Local**, an open-source benchmarking tool designed to measure how fast AI agents perform on laptops and workstations. The tool replays real agent trajectories—eight recorded tasks spanning 168 model turns—with a context window growing to ~56K tokens. It isolates inference speed by default but can also simulate tool delays or live CPU calls.\n\nThe initial results cover four hardware setups: **NVIDIA DGX Spark (128GB)**, **RTX 5090 (32GB)**, **AMD Ryzen AI Halo (128GB)**, and **MacBook Pro M5 Pro (64GB)**. Benchmarks focus on models like **Qwen3.5-9B, Qwen3.8-27B, Qwen3.6-35B-A3B, and Ling 3.0 Flash (124B/5B active)**, all tested at 4-bit quantization. The **RTX 5090** emerged as the fastest system for models fitting in its memory, while the **DGX Spark** outperformed the Ryzen AI Halo due to higher compute efficiency. The MacBook Pro (M5 Pro) showed competitive results, finishing within 2–21% of the Ryzen AI Halo for some models. The tool also highlights how prefill speeds—reading new input tokens—can dominate latency, especially on systems with lower compute-to-memory bandwidth ratios. Future updates will expand hardware coverage, add multi-agent scenarios, and include user-submitted leaderboards.","keyPoints":["AA-AgentPerf-Local benchmarks local AI agent performance on laptops/workstations with 8 recorded tasks (~56K tokens)","RTX 5090 fastest for models fitting in 32GB; DGX Spark outperforms Ryzen AI Halo due to compute efficiency","Tool supports any OpenAI-compatible server and will expand to multi-agent scenarios and live CPU tool-calling"],"whyItMatters":"Helps developers and businesses compare hardware and model setups for local AI deployment, reducing trial-and-error costs. Useful for small teams running agents on laptops or workstations without cloud dependencies.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":["Artificial Analysis"],"models":["Qwen3.5-9B","Qwen3.8-27B","Qwen3.6-35B-A3B","Ling 3.0 Flash"],"people":[]},"firstPublishedAt":"2026-09-30T08:41:50Z","updatedAt":"2026-09-30T08:41:50Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"artificialanalysis.ai","title":"AA-AgentPerf-Local: Benchmarking local AI agents on laptops and workstations","url":"https://artificialanalysis.ai/articles/aa-agentperf-local","publishedAt":"2026-09-30T08:41:50Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/LocalLLaMA/comments/1wtzpcr/aaagentperflocal_benchmarking_local_ai_agents_on/","points":null}],"thread":{"title":"Artificial Analysis AI Benchmarking Initiative Chronicle","url":"https://digestai.news/thread/artificial-analysis-launches-cyber-index-alliance-to-benchmark-ai-agents-on","storyCount":2},"cite":{"text":"Digest AI, \"Artificial Analysis releases open-source tool to benchmark local AI agents on laptops and workstations\", 30 September 2026, https://digestai.news/story/artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag","publisher":"Digest AI","title":"Artificial Analysis releases open-source tool to benchmark local AI agents on laptops and workstations","datePublished":"2026-09-30T08:41:50Z","url":"https://digestai.news/story/artificial-analysis-releases-open-source-tool-to-benchmark-local-ai-ag"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}