Benchmarks/DataCenterBench/Flip the light switch

Flip the light switch

DataCenterBenchRig 6
Robot
I2RT YAM, two-arm
Trials
3

Flip the lever of the light switch on the white box to the other side.

Successful run · Grok 4.7Source ↗
RubricPass/fail
0The lever is on the same side as at the start.
1The lever is on the other side of its slot compared with the start.
Full criteria and notes
Full success check
1 if: The lever is on the other side of its slot compared with the start. Else 0. two LLM judges (GPT-6 Astra, Claude Opus 5.5) on start/end photos must agree, else a person scores it.
Trials
3 per model (27 counted over 9 models); avg@3.

Results on this task

Task definition ↗ · All DataCenterBench tasks