Turn the key
- Robot
- I2RT YAM, two-arm
- Trials
- 3
Turn the key clockwise to its other position. Leave it there and release it.
Successful run · Gemini 4 ArgonSource ↗
RubricPartial credit
0The key is in its start position (or turned the other way).
0.5The key is turned to its other position but the gripper is still holding it.
1The key is turned clockwise to its other position and left there, with the gripper released.
Full criteria and notes
- Full success check
- 1 if: The key is turned clockwise to its other position and left there, with the gripper released. 0.5 if: The key is turned to its other position but the gripper is still holding it. Else 0. two LLM judges (GPT-6 Astra, Claude Opus 5.5) on start/end photos must agree, else a person scores it.
- Trials
- 3 per model (27 counted over 9 models); avg@3.
Results on this task
ModelKeyCost / attemptTime / attempt
Gemini 4 Argon100$1.8139 min 53 s
GPT-6 Astra33.3$19.253 min 36 s
GPT-6.1 Sol33.3$10.562 min 55 s
Claude Opus 5.533.3$19.1015 min 36 s
GLM 5.3 Flash33.3$0.00808 min 56 s
Kimi K333.3$0.43013 min 23 s
Grok 4.733.3$0.5454 min 42 s
Muse Wonder 1.00.00$1.999 min 53 s
DeepSeek V4.1 Flash0.00$0.1447 min 14 s
DataCenterBench