Robo Use

Close the drawer in RoboCasa kitchen layout 2 (Franka Panda on a mobile base)

robocasa-close-drawer-kitchen2RoboCasaFranka Panda on a mobile baseManipulationMedium difficulty

Reference solution, 293 of 400 steps.

Instruction

A household kitchen from RoboCasa (kitchen layout 2, style 2): counters, cabinets, drawers, a sink and appliances, simulated in MuJoCo. The robot is a Franka Panda arm on a mobile base (RoboCasa's PandaOmron), parked where RoboCasa's task set-up puts it, facing the relevant fixture; the base and torso do not move in this task, only the arm and its two-finger gripper (fingers open to 8 cm). World frame in metres, z up; robot_forward in robo observe is the horizontal direction the robot faces. The arm's reach from the parked base is limited (about 0.7 m forward of robot_base_pos at counter height, less low down or far to the side), and the arm and hand can collide with the counter and fixtures.

Goal: One of the drawers next to the robot is pulled open. Close it fully.

Success: RoboCasa's own check for CloseDrawer: the drawer is at most 5 % open (drawer_open_fraction <= 0.05, where 1 means pulled out by 55 % of the drawer's depth, as far as RoboCasa opens drawers). Success is judged by the episode server after you call robo done and the robot has held still for about 10 steps.

Task fields in robo observe: drawer_handle_pos (centre of the drawer handle; it moves with the drawer), drawer_open_fraction (0 closed .. 1 open as far as RoboCasa opens drawers) and drawer_front_normal (unit vector pointing out of the drawer front, toward where it opens).

Controls. robo act DX DY DZ GRIP moves the gripper's commanded target by DX, DY, DZ times 2 cm per step along the WORLD x, y, z axes (each in [-1, 1]); the gripper keeps its orientation (pointing straight down). GRIP 0 keeps the fingers as they are, any positive value closes them, and a negative value -f opens them to fraction f (-1 fully open). One step is 50 ms (RoboCasa's 20 Hz control rate). The arm follows the target through RoboCasa's operational-space controller; the target stays within 5 cm of the hand. robo move-to X Y Z and robo grip G work as described below.

Observation. robo observe reports hand_pos (the point between the fingertips), hand_target (the commanded target), gripper_open (0 closed .. 1 open), robot_base_pos, robot_forward, task_instruction (RoboCasa's own language instruction) and the task fields listed above. robo observe --image saves a picture from the robot's shoulder camera (--camera robot0_agentview_left, robot0_agentview_right, robot0_eye_in_hand and robot0_frontview are also available).

The step budget is 400 steps (20 s of simulated time).

How the robot is controlled and scored

How to control the robot

You are the robot's policy. You act only through the robo command in your shell. There is no other way to move the robot, and you cannot read or change the simulator, the scoring, or other files to succeed; the episode server judges the final physical state itself.

robo info                         # action space, available skills, step budget
robo observe                      # robot and object state as numbers
robo observe --image              # also saves a camera image and prints its path (open it to look)
robo act DX DY DZ GRIP [--repeat N]   # low-level action, applied N times (N <= 50)
robo move-to X Y Z [--grip G]     # skill: move the gripper toward a point (if enabled for this task)
robo grip G [--steps N]           # skill: hold position and set the gripper (+1 close, -1 open)
robo done "short summary"         # end the episode and ask for scoring
robo give-up "reason"             # end the episode without claiming success
  • Positions are in metres in the world frame (x, y on the table plane, z up).
  • The episode has a fixed step budget (see robo info); every simulated step counts, including skills.
  • Unless the task says otherwise, success is judged about 10 steps after you call robo done, with the robot holding still, so the goal must still be true when the robot stops.
  • Work in small steps and re-observe after each motion. Call robo done exactly once when finished.

Run this task

robouse run \
  --task robocasa-close-drawer-kitchen2 \
  --harness claude-code \
  --out runs/robocasa-close-drawer-kitchen2

Needs RoboCasa's own virtualenv, about 4.7 GB first.

The robocasa-close-drawer-kitchen2 task folder in Robo Use 0.6.1. The task text is shown here; the reference solution and the verifier are on GitHub.