Benchmarks/Embodied First Exam/Single-stage · TurnOffSinkFaucet

Single-stage · TurnOffSinkFaucet

Embodied First ExamRoboCasa
Robot
Franka on Omron base
Trials
1
Time limit
1 h 00 min

RoboCasa (TurnOffSinkFaucet, a Franka Panda arm on an Omron mobile base (with a torso lift)): Turn off the sink faucet. Produce one replayable trajectory of the robot's native controller commands on a frozen instance; graded on the state after the last action.

Official Embodied First Exam clipSource ↗
SuccessPass/fail

Pass or fail: the simulator checks the goal described above.

Full criteria and notes
Full success check
What TurnOffSinkFaucet's check requires (RoboCasa @ 756598a), all on that final state:. The water is off: the faucet handle's angle on its on / off hinge (mod 360°) is outside 0.40 rad (about 23°) to π from its off position, i.e. turned back toward 0 to below 0.40 rad (it starts at about 0.47 rad; the handle turns on two hinges at one pivot: the on / off hinge, whose range is 0 to 0.52 rad (so 0.40 rad is near the end of its travel), and a temperature hinge (about ±0.79 rad) that moves the handle in another direction and never turns the water on or off; which way each hinge turns depends on the sink model; what counts is that hinge's angle, not how high the handle is).
Scoring
A simulator error or a missing result counts as a fail.
Trials
1 graded attempt per test (standard mode: one live episode, no replay).

Graded example

Turn off the sink faucet · standard mode

Pass · Claude Opus 5.5: pass. 5,174 control steps, 1,737 s.
Fail · GPT-6 Astra: fail. Ran to the 3,600 s cutoff.
AgentStandard
Claude Opus 5.5Pass
Gemini 4 ArgonPass
Kimi K3.1 ProPass
DeepSeek V4.1 FlashPass
GPT-6 AstraFail
GLM 5.3 FlashFail
Qwen3.8 MaxFail
Gemini 3.8 FlashFail

Check: the faucet handle ends outside 0.40–π rad after the last step, and the task was not already solved at the start.

Task definition ↗ · All Embodied First Exam tasks