Elorian and Scale AI jointly released the Humanity’s Sixth Sense (HSS) benchmark, designed to measure how models interpret and reason about information a visual scene implies but does not explicitly show. The project focuses on intuitive visual reasoning: tasks that rely on implied facts about a scene rather than explicit visual features.
Task types and dataset details
HSS contains 522 questions based on 288 images and 234 short video clips; the videos total 17.6 hours. Questions are grouped into four categories:
- Temporal & causal dynamics: “What happened before the scene, and what happens next?”
- Physical & spatial logic: “What fits, what’s within reach, and what someone else can see from where they stand?”
- Social understanding: “Intent, attention, belonging.”
- Abstract & contextual inference: “The patterns and rules a scene implies.”
Example tasks include judging whether more books can fit on a photographed bookcase or deciding if chains in a photo of horses pulling a plough bear the plough’s drag weight.
Performance: humans vs. models
The human baseline is high though not perfect: aggregate pass@1 for humans is 93.1. Category breakdown: image 94.1; video 91.9; time (temporal) 97; space 94.9.
The best-performing AI system reported is GPT 6 Astra, with an aggregate pass@1 of 53.6. Its category scores are: image 55.6; video 51.1; time 58.2; and space 53.6. Some Chinese open-weight models perform substantially worse: GLM-5.3-flash scores 26.8 pass@1, and Kimi K3 scores 25.5 pass@1.
The benchmark authors note the two most common single mistakes: (1) missing a detail that determines the answer (21% of failures), and (2) misidentifying an object, person, or role (20% of failures).
Why these results matter
HSS can be read in two ways. On one hand, it highlights current systems’ shortcomings in intuitive visual reasoning. On the other hand, it shows how capable today’s frontier multimodal large language models (LLMs) are at tasks many thought would require specialized, spatially aware 2D/3D world models.
The fact that LLM-based multimodal systems such as the GPT series and Claude perform as well as they do on HSS challenges the idea that true visual reasoning necessarily requires explicit 3D world models. At the same time, the substantial gap to human performance underscores remaining limitations: models still fail frequently due to missed fine details and incorrect identifications.
Summary
Humanity’s Sixth Sense is a 522-question benchmark using 288 images and 234 video clips to probe intuitive visual reasoning. Humans score 93.1 pass@1, while the top AI (GPT 6 Astra) reaches 53.6, with open-weight models often far below that. The benchmark illuminates concrete failure modes and raises questions about how and whether world-model-like capabilities emerge in multimodal LLMs.
References
- Introducing Humanity’s Sixth Sense (HSS) — Elorian
- Humanity’s Sixth Sense: Benchmarking Intuitive Visual Reasoning in Multimodal Models — Scale Labs
