Flip the light switch
- Robot
- I2RT YAM, two-arm
- Trials
- 3
Flip the lever of the light switch on the white box to the other side.
Successful run · Grok 4.7Source ↗
RubricPass/fail
0The lever is on the same side as at the start.
1The lever is on the other side of its slot compared with the start.
Full criteria and notes
- Full success check
- 1 if: The lever is on the other side of its slot compared with the start. Else 0. two LLM judges (GPT-6 Astra, Claude Opus 5.5) on start/end photos must agree, else a person scores it.
- Trials
- 3 per model (27 counted over 9 models); avg@3.
Results on this task
ModelSwitchCost / attemptTime / attempt
GPT-6 Astra100$20.763 min 49 s
GPT-6.1 Sol100$4.281 min 40 s
Grok 4.766.7$0.5433 min 23 s
Claude Opus 5.533.3$16.2313 min 46 s
Muse Wonder 1.033.3$1.368 min 06 s
Gemini 4 Argon0.00$0.5499 min 47 s
GLM 5.3 Flash0.00$0.01715 min 16 s
Kimi K30.00$0.60722 min 37 s
DeepSeek V4.1 Flash0.00$0.1287 min 40 s
DataCenterBench