Benchmarks/AA-Robotics

AA-Robotics

Real robotsPass/fail + partialReleased Oct 2026Index weight 0.6
Artificial AnalysisMaintainer ↗

8 tasks, press a button, place a toy, pour a drink, tidy blocks, match shapes, write hello world, build a tower and measure a box.

As of November 17, 2026, GPT-6.2 Astra leads with 79.0, followed by GPT-6.2 Sol (78.0) and Claude Fable 5.2 (75.0). Official leaderboard ↗

Leaderboard

Tasks
8
Attempts / task
1
Median cost / task
$7.61
Median time / task
31 min 12 s

Overall vs speed

Published time per task attempt

Faster, higher scoring50.060.070.025 min 00 s33 min 20 s50 min 00 sTime per task (faster ←)Overall (better ↑)GPT-6 AstraGPT-6.1 Sol

Overall vs cost

Published cost per task attempt

Cheaper, higher scoring50.060.070.0$2.00$5.00$10.00$20.00Cost per task (cheaper ←)Overall (better ↑)GPT-6 AstraGPT-6.1 Sol

Task score 0 to 100, one attempt per task, judged by a person. Overall scores for GPT-6.2 Astra, GPT-6.2 Sol, Claude Fable 5.2, Gemini 4 Hydrogen and Gemini 4 Carbon come from a chart in Artificial Analysis's November 17 GPT-6.2 article, which gives no harness, effort setting or task scores. They are held, not counted in the index, until Artificial Analysis publishes the runs.

#Model
1GPT-6.2 AstraNot counted in the index79.0—————
2GPT-6.2 SolNot counted in the index78.0—————
3Claude Fable 5.2Not counted in the index75.0—————
4GPT-6 Astra69.4100100100$16.3630 min 18 s5GPT-6.1 Sol64.61001000.00$1.9328 min 06 s
6Gemini 4 HydrogenNot counted in the index58.0—————
7Gemini 4 Argon55.3100100100$7.6152 min 24 s8Claude Opus 5.548.810010025.0$6.7531 min 42 s9Claude Fable 5.146.310010030.0$9.7331 min 12 s
10Gemini 4 CarbonNot counted in the index43.0—————

Tasks

8 tasks. Hover a task to play its clip (8 of 8).

8 tasks · 8 with clips

Methodology

Scoring
Pass/fail + partial
Time cap / attempt
—
Control rate
—
Models in index
5 of 10

Scoring. A human judge awards 100 for the pass criterion, 0–99 for partial progress.

Who runs it. Independent evaluator (Artificial Analysis), one human-judged attempt per task.

Robots. Hiwonder NexArm (one arm).

Note. Coding agents only. Research preview.

Time per attempt. Published or measured for 5 of 5 models.

Cost per attempt. Published for 5 of 5 models.