Benchmarks/VLA-Replica/Put bread on plate

Put bread on plate

VLA-ReplicaIn-distribution · Pick-and-Place
Robot
SO-101
Trials
15
Time limit
1 min 30 s

Put the bread on the red plate.

Official demo · 1.8× speedSource ↗
SuccessPass/fail

Bread is resting on the correct colored plate, and SO-101 moved back to its home position.

Full criteria and notes
Instruction variants
Easy: Put the bread on the red plate. · Easy: Put the bread on the blue plate. · Medium: Put the bread on the blue plate. · Medium: Put the bread on the red plate. · Hard: Put the bread on the red plate.
Scoring
Pass/fail per trial, operator-judged, 90 s limit.

Results on this task

Graded example

Put bread on plate · 15 trials per model

Reference scene (SO-101).
ModelSuccess± SD
MolmoAct20.930.12
π0.50.800.20
GR00T N1.70.800.00
SmolVLA0.800.20
π00.670.12
ACT0.330.12
X-VLA0.270.12

Pass when the bread rests on the correct coloured plate and the arm has returned home.

Task definition ↗ · All VLA-Replica tasks