StarSkirmish Bench v0.2 ranks GPT-6 Astra as top StarCraft bot writer
StarSkirmish Bench v0.2 tested how well leading AI models could write StarCraft: Brood War bots in one hour. Each model generated a Protoss bot in C++, which was then evaluated against a roster of competitive human-written bots and demo bots. The benchmark measures long-horizon reasoning and agentic coding, as models must adapt based on practice games to improve their bots.
Key points
- GPT-6 Astra tops StarSkirmish Bench v0.2 with a 39% win rate against human-written bots
- Gemini 4 Argon and Claude models score 37–39%, trailing GPT-6 Astra by a clear margin
- Human-written bots still dominate: GPT-6 Astra wins only 7% of games against top human bots
GPT-6 Astra from OpenAI scored highest with an average win rate of 39%, surpassing GPT-6.1 Sol in second place. Gemini 4 Argon, Claude Opus 5.5, and Claude Fable 5.1 tied for third, each scoring between 37 and 39. Despite these results, the strongest human-written bots still outperform AI-generated ones significantly—GPT-6 Astra won only 7% of games against the top human bots. The benchmark also highlighted variability in model performance, with Gemini 4 Argon showing the widest range in win rates across its five runs.
Model pages: GPT-6-Astra → · GPT-6.1 Sol → · Gemini 4 Argon →
The story so far
5 episodes →- StarSkirmish Bench v0.2 ranks GPT-6 Astra as top StarCraft bot writerthis story
Show HN: Gemini 4 Argon plays StarCraft
starskirmish.com · 16 October 2026
Loading the full article…
This text was published by starskirmish.com. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Hacker News discussion · 41 pointsnews.ycombinator.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Former OpenAI researcher says new theory may explain AI thinking, could affect safety · 2 src
- Author builds CodeOwnership tool to check AI-generated code ownership · 1 src
- Nvidia pledges $250M in compute for UK AI agent safety research · 11 src
- Agentic RAG adds reasoning loops to retrieval systems for complex queries · 1 src
- OpenAI offers U.S. university labs free ChatGPT Business seats with Rosalind access · 3 src
Comments
via GitHub Discussions