Benchmarks/MolmoSpaces

MolmoSpaces

SimulationCode checkReleased Feb 2026Index weight 0.72
Ai2Maintainer ↗

Pick, pick and place, open, close, randomized camera and commonsense tasks.

As of November 17, 2026, MagicBrain + Qwen3.7-Plus leads with 94.4, followed by MolmoBot-Img (92.8) and MolmoBot (92.0). Official leaderboard ↗ · Paper ↗

Leaderboard

Tasks
19
Attempts / task
1
Median cost / task
$0.018
Median time / task
20 s

Success at any point in the episode (%). Blank means the model did not run that task.

#Model
1MagicBrain + Qwen3.7-PlusNot counted in the index94.484.727.782.7——
2MolmoBot-ImgNot counted in the index92.8—————
3MolmoBot92.0———$0.015est.16 sest.
4MolmoBot-f3Not counted in the index91.9—————
5Awomo-0.5V86.458.962.486.0$0.015est.17 sest.
6EchoPolicy-0 + Gemini-3.8-FlashNot counted in the index84.755.670.291.3——
7Awomo-0.5I84.571.049.387.2$0.016est.17 sest.
8PrimeR0Not counted in the index77.554.627.881.3——
9TGQWENNot counted in the index74.8—————
10WALL-OSS-0.5Not counted in the index73.846.016.878.5——
11Phoenix + Gemini-ERNot counted in the index73.743.837.350.0——
12Psi-R2Not counted in the index73.141.59.062.2——
13AsterPickNot counted in the index72.3—————
14TiPToP + Gemini-ERNot counted in the index68.733.2————
15Cosmos3-Nano-Policy65.535.232.079.1$0.018est.20 sest.
16MolmoBot-π₀Not counted in the index64.0—————
17Cosmic + Gemini-ERNot counted in the index62.644.758.587.9——
18Magic-PolicyNot counted in the index60.5—————
19Vana-PolicyNot counted in the index59.736.135.160.8——
20CAPpick + Gemini-ERNot counted in the index55.4—————
21PRTS-DroidNot counted in the index53.827.712.076.3——
22PAT + Qwen3.8-27BNot counted in the index51.731.322.481.3——
23MolmoAct248.323.311.771.3$0.021est.23 sest.
24π₀.₅-MolmoSpacesNot counted in the index45.0—————
25PAT-RoboLab + Qwen3.8-27BNot counted in the index42.424.717.063.0——
26π0.536.413.622.765.1$0.022est.24 sest.
27LAP-VLANot counted in the index24.96.611.445.9——
28π0-FAST22.010.911.138.6$0.024est.26 sest.29π016.212.511.053.1$0.025est.27 sest.
30StereoVLANot counted in the index7.0—————
—CAPclose + Gemini-ERNot counted in the index———83.8——
—CAPopen + Gemini-ERNot counted in the index——41.9———
—DreamZero———————X-VLA——————
—ZoeInside-VLANot counted in the index——36.685.7——
—π₀.₅-MolmoBot-FTNot counted in the index——————

Tasks

19 tasks. Hover a task to play its clip (8 of 19).

19 tasks · 8 with clips

Methodology

Scoring
Code check
Time cap / attempt
30 s
Control rate
15.2 Hz
Models in index
8 of 36

Scoring. Simulator pass/fail check, e.g. object lifted 1 cm and held only by the robot.

Who runs it. Ai2 plus submitted results.

Robots. Simulated Franka FR3.

Note. Its combined score counts tasks a model skipped as zero, so we use single tasks.

Time per attempt. Not published. Estimated otherwise: time cap 30 s × (1 − success/2).

Cost per attempt. Not published. Estimated otherwise: one H100 at $3.29 per hour (Lambda on-demand, Oct 2026) × time. GPU price source ↗