RoboArena
RoboArena teamMaintainer ↗Open-ended tasks chosen by evaluators, mostly pick and place. Ranked by pairwise preference.
As of November 17, 2026, Spirit v1.6 (j2-vla) leads with 1788, followed by DreamZero (1735) and Apricot (1661). Official leaderboard ↗ · Paper ↗
Leaderboard
- Tasks
- 2,752
- Attempts / task
- —
- Median cost / task
- $0.023
- Median time / task
- 25 s
Rating vs speed
Published time per task attempt
Cost isn’t published
RoboArena doesn’t report cost per attempt for enough models to chart. The table shows Playground’s estimates, marked est.; Methodology explains how they’re made.
Head-to-head rating from blind comparisons. Entries under 100 sessions are low-sample on the site.
DreamZero173542.6190$0.023est.25 s
π0.5160830.7745$0.024est.26 s6
π0-FAST158229.6941$0.023est.25 s7
MolmoAct215791955.0——
GalaxeaVLA G0.5155969.545.0$0.018est.20 sest.
π0146129.81120$0.023est.25 s
Phoenix12221968.0——24
PaliGemma-binning78650.0629$0.019est.21 sTasks
2,752 evaluator-written instructions; the 30 most-run shown. Hover a task to play its clip (30 of 30).
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ ClipGraded examples
“Cover the book with the blue towel” · one A/B session
Evaluator preference: B. Neither finished; only the preference enters the ranking (Bradley-Terry with ties).
The session counts as a loss for DreamZero, which still rates higher overall (1,735 vs 1,608).
Methodology
- Scoring
- Pairwise preference
- Time cap / attempt
- 27 s
- Control rate
- 15 Hz
- Models in index
- 6 of 24
Scoring. Evaluators run two hidden policies on a task of their choosing and pick A, B or tie.
Who runs it. Volunteer labs, blind head-to-head comparisons.
Robots. Franka (DROID setup).
Note. Entries with 20 or more sessions. The site's main board needs 100.
Time per attempt. Published or measured for 5 of 6 models. Estimated otherwise: time cap 27 s × (1 − success/2).
Cost per attempt. Not published. Estimated otherwise: one H100 at $3.29 per hour (Lambda on-demand, Oct 2026) × time. GPU price source ↗