Benchmarks/RoboColosseum Open

RoboColosseum Open

Real robotsProgress + preferenceReleased Nov 2026Index weight 0.722
RoboColosseum teamMaintainer ↗

Fresh instructions written on the spot at each site, in offices, homes, parks and factories.

As of November 17, 2026, GPT-6 Astra leads with 1925, followed by MolmoAct2 (1869) and GalaxeaVLA G0.5 (1744). Official leaderboard ↗ · Paper ↗

Leaderboard

Tasks
7,137
Attempts / task
—
Median cost / task
$0.036
Median time / task
40 s

Difficulty-calibrated rating on an Elo-style scale, fitted separately for each robot, so compare within a column only.

Tasks

7,137 evaluator-written instructions; the 30 most-run shown.

30 tasks · 0 with clips

Methodology

Scoring
Progress + preference
Time cap / attempt
53 s
Control rate
15 Hz
Models in index
8 of 9

Scoring. Evaluators write the task, score each blinded run 0–100 and pick A, B or tie.

Who runs it. Operator network at 124 everyday sites; blinded evaluators pick the better of two unnamed runs.

Robots. Franka (DROID setup), two-arm YAM, SO-101.

Note. Rated per robot from task success, adjusted for task difficulty; robots are never compared with each other. MolmoAct2's authors help run it. Not the same benchmark as RoboColiseum.

Time per attempt. Not published. Estimated otherwise: time cap 53 s × (1 − success/2).

Cost per attempt. Not published. Estimated otherwise: GPT-6 Astra's median published cost per attempt on other boards; Claude Opus 5.5's median published cost per attempt on other boards; one H100 at $3.29 per hour (Lambda on-demand, Oct 2026) × time; Grok 4.7's median published cost per attempt on other boards. GPU price source ↗