10 tasks in pick and place, object interaction and counting, plus an out-of-distribution track.
As of November 17, 2026, π0.5 leads with 53.0, followed by GR00T N1.7 (42.0) and MolmoAct2 (39.0). Official leaderboard ↗ · Paper ↗
Leaderboard
- Tasks
- 18
- Attempts / task
- 15
- Median cost / task
- $0.069
- Median time / task
- 76 s
Success rate (%), 15 trials per task over three rounds. Task columns are the in-distribution track.
π0.553.039.080.087.0$0.060est.66 sest.2
GR00T N1.742.035.080.087.0$0.065est.71 sest.3
MolmoAct239.047.093.073.0$0.066est.72 sest.4
π032.030.067.067.0$0.069est.76 sest.5
SmolVLA29.027.080.020.0$0.070est.77 sest.6
ACT21.08.033.07.0$0.074est.81 sest.7
X-VLA13.05.027.013.0$0.077est.84 sest.Tasks
18 tasks. Hover a task to play its clip (18 of 18).
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ Clip
▶ ClipGraded examples
Put bread on plate · 15 trials per model

| Model | Success | ± SD |
|---|---|---|
| MolmoAct2 | 0.93 | 0.12 |
| π0.5 | 0.80 | 0.20 |
| GR00T N1.7 | 0.80 | 0.00 |
| SmolVLA | 0.80 | 0.20 |
| π0 | 0.67 | 0.12 |
| ACT | 0.33 | 0.12 |
| X-VLA | 0.27 | 0.12 |
Pass when the bread rests on the correct coloured plate and the arm has returned home.
Methodology
- Scoring
- Pass/fail
- Time cap / attempt
- 1 min 30 s
- Control rate
- 30 Hz
- Models in index
- 7 of 9
Scoring. Operator checks a one-line success criterion per trial.
Who runs it. UT Dallas IRVL runs public policies, plus submissions.
Robots. SO-101 (one arm).
Note. 15 trials per task. Every policy is fine-tuned on the same 50 demonstrations per task.
Time per attempt. Not published. Estimated otherwise: time cap 90 s × (1 − success/2).
Cost per attempt. Not published. Estimated otherwise: one H100 at $3.29 per hour (Lambda on-demand, Oct 2026) × time. GPU price source ↗
