Benchmarks/ArmnetBench

ArmnetBench

Real robotsOperator labelReleased Jul 2026Index weight 0.765
ArmnetMaintainer ↗

12 tasks, 8 one-arm and 4 two-arm, from block stacking to cable clips and towel folding.

As of November 17, 2026, π0.5 leads with 47.6, followed by π0 (35.1) and GR00T N1.7 (29.4). Official leaderboard ↗ · Paper ↗

Leaderboard

Tasks
12
Attempts / task
30
Median cost / task
$0.025
Median time / task
27 s

Overall vs speed

Published time per task attempt

Faster, higher scoring20.030.040.050.015 s20 s30 sTime per task (faster ←)Overall (better ↑)π0.5GR00T N1.7

Cost isn’t published

ArmnetBench doesn’t report cost per attempt for enough models to chart. The table shows Playground’s estimates, marked est.; Methodology explains how they’re made.

Strict success (%), 30 rollouts per task, about 360 per policy.

Tasks

12 tasks. Hover a task to play its clip (12 of 12).

12 tasks · 12 with clips

Methodology

Scoring
Operator label
Time cap / attempt
—
Control rate
20 Hz
Models in index
7 of 7

Scoring. Operator labels each rollout successful, suboptimal or failure.

Who runs it. Armnet runs every rollout on its arm farm.

Robots. SO-101 (one-arm and two-arm cells).

Note. Every policy is trained by Armnet on the same 50 demonstrations per task. Paper results, no live board.

Time per attempt. Published or measured for 7 of 7 models.

Cost per attempt. Not published. Estimated otherwise: one H100 at $3.29 per hour (Lambda on-demand, Oct 2026) × time. GPU price source ↗