# Elorian and Scale AI release benchmark showing AI lags in visual reasoning

Digest AI · Research · published 2026-10-12T12:30:39Z

Canonical: https://digestai.news/story/elorian-and-scale-ai-release-benchmark-showing-ai-lags-in-visual-reaso

## Summary

Elorian and Scale AI introduced **Humanity’s Sixth Sense**, a benchmark testing how AI models interpret implied details in visual scenes. The test covers 522 questions across 288 images and 234 short video clips (17.6 hours total), divided into categories like physical logic, social understanding, and abstract inference.

Human performance is high—93.1% pass rate—but the best AI model, **GPT-6 Astra**, scores only 53.6%. Chinese open-weight models like **GLM-5.3-flash** and **Kimi K3** perform worse, with pass rates of 26.8% and 25.5%, respectively. The authors note that AI often misses critical details or misidentifies objects, highlighting gaps in intuitive visual reasoning. The paper argues this challenges assumptions that advanced world models are needed for such tasks.

## Key points

- Benchmark tests AI’s ability to infer implied details in visual scenes, with 522 questions across images and videos
- GPT-6 Astra scores 53.6% pass rate, far below human baseline of 93.1%, while Chinese models lag further
- Common AI failures include missing key details (21% of errors) and misidentifying objects (20%)

## Why it matters

The benchmark exposes limitations in AI’s ability to reason about visual contexts, which could slow progress in applications like robotics, autonomous systems, or AI-assisted design. It also suggests current models may overperform on tasks requiring less nuanced spatial or social understanding.

## Sources

1. [Import AI 476: Creativity and RSI; worse-than-China open weight models; AI goes to space](https://jack-clark.net/2026/10/12/import-ai-476-creativity-and-rsi-worse-than-china-open-weight-models-ai-goes-to-space) (Import AI, 2026-10-12)

## Cite

Digest AI, "Elorian and Scale AI release benchmark showing AI lags in visual reasoning", 12 October 2026, https://digestai.news/story/elorian-and-scale-ai-release-benchmark-showing-ai-lags-in-visual-reaso

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/elorian-and-scale-ai-release-benchmark-showing-ai-lags-in-visual-reaso.json