Pick with Object Permanence
- Robot
- Franka FR3
- Trials
- 1,000
- Time limit
- 20 s
- Step limit
- 300
Pick up the mug that the ball is hidden under (mugs fall at scene start, one covering the ball).

No public clip for this task
SuccessPass/fail
PickTask success evaluated on the correct mug, the one covering the ball. Mapping inferred from names.
Full criteria and notes
- Full success check
- PickTask success (lifted >= threshold, only robot contact) evaluated on the correct mug, the one covering the ball (MugBallPickTask). Mapping inferred from names. Episode counts as success if this holds at any step (oracle success, the ranking metric); at-end success is also reported for some tasks. Benchmark: molmospaces-commonsense-bench; max steps 300.
- Scoring
- 1 per episode; task score = oracle successes / episodes x100.
- Trials
- ~1,000 distinct episodes, each run once (median n=1000; varies by entry).