Benchmarks/MolmoSpaces/Pick & Place (MB-Pick & Place)

Pick & Place (MB-Pick & Place)

MolmoSpacesMolmoBot v1.5/v2
Robot
Franka FR3
Trials
1,000
Time limit
30 s

Pick up the object and place it in/on the receptacle.

Official MolmoBot clip · 2.8× speedSource ↗
SuccessPass/fail

Object supported by the target receptacle, no robot-object contact, receptacle moved at most 0.10 m and tilted at most 45 deg.

Full criteria and notes
Full success check
Object supported by the target receptacle (contact force carrying at least 50% of its weight, with an on-receptacle fallback and a pose carry-forward), no robot-object contact, receptacle moved at most 0.10 m and tilted at most 45 deg (PickAndPlaceTask.get info). Episode counts as success if this holds at any step (oracle success, the ranking metric); at-end success is also reported for some tasks. Benchmark: molmospaces-bench-v2/procthor-objaverse/FrankaPickandPlaceHardBench (20260206), filament.
Scoring
1 per episode; task score = oracle successes / episodes x100.
Trials
~1,000 distinct episodes, each run once (median n=1000; varies by entry).

Results on this task

ModelPick and place
MagicBrain + Qwen3.7-Plus84.7
Awomo-0.5I71.0Awomo-0.5V58.9
EchoPolicy-0 + Gemini-3.8-Flash55.6
PrimeR054.6
WALL-OSS-0.546.0
Cosmic + Gemini-ER44.7
Phoenix + Gemini-ER43.8
Psi-R241.5
Vana-Policy36.1
Cosmos3-Nano-Policy35.2
TiPToP + Gemini-ER33.2
PAT + Qwen3.8-27B31.3
PRTS-Droid27.7
PAT-RoboLab + Qwen3.8-27B24.7
MolmoAct223.3π0.513.6π012.5π0-FAST10.9
LAP-VLA6.6

Task definition ↗ · All MolmoSpaces tasks