The Story

The ARC Prize Foundation, led by François Chollet, has introduced ARC-AGI-3, the latest version of its interactive reasoning benchmark designed to test general intelligence in AI systems.

According to details on the official page at ARC Prize, the new benchmark focuses on tasks that humans can typically solve immediately, while current AI systems struggle significantly.

What makes ARC-AGI-3 different

• No instructions: Agents are placed in game-like environments with no prior guidance.
• Discovery-based reasoning: Systems must identify patterns, form goals, and plan strategies from scratch.
• Generalization focus: The benchmark aims to measure true reasoning ability rather than memorization or pattern matching.

Current model performance

• Low scores across the board: Leading AI models score below 1% on ARC-AGI-3.
• Top results: Gemini Pro leads with ~0.37%, followed by GPT-5.4 (~0.26%) and Opus 4.6 (~0.25%).
• Human baseline: Humans are able to solve these tasks reliably on the first attempt.

Context from earlier versions

Previous versions of the ARC benchmark saw rapid improvements as labs trained models specifically for the test, with scores rising from low single digits to around 50% on earlier iterations.

ARC-AGI-3 is designed to reset that progress and better evaluate whether models can genuinely reason rather than optimize for specific benchmarks.

Why it matters

The release highlights a key gap between current AI systems and human-level reasoning. While models have made rapid progress in language and coding tasks, benchmarks like ARC-AGI-3 suggest that general problem-solving remains a major challenge.

For the industry, it reinforces the idea that scaling alone may not be sufficient, and that new approaches to reasoning and learning may be required to move closer to general intelligence.