Lift a target cylinder out of a ring of protected neighbours into a tray (ALOHA 2)

An ALOHA 2 bimanual rig: two ViperX 300 arms (6 joints each, with parallel-jaw grippers that open to about 7 cm) are mounted side by side at the back edge of a 1.6 m x 0.94 m desk, both reaching forward toward you. The "left" arm is the one at negative x (on the left in the front camera view), the "right" arm the one at positive x. This is a MuJoCo physics simulation with the MuJoCo Menagerie robot models: every object is a free rigid body that moves only through contact and friction (nothing is attached or teleported), so a loose grip, a fast swing or a collision can drop or knock things over. World frame in metres: +x to the right, +y toward the robots (the back of the desk), +z up; the desk top is z = 0.

Goal: In the right bowl a red cylinder (target, 2.4 cm across, 2.8 cm tall) stands in a ring of four violet cylinders of the same size (neighbour_0 .. neighbour_3, 4.5 cm from it, centre to centre). Lift the red cylinder out without disturbing the violet ones and set it upright in the yellow tray in the middle of the desk.

Success: The target's base centre is within 15 mm of tray_center, it rests on the tray floor (within 4 mm of z = 0.004), is tilted at most 12 degrees, released and at rest; and at no time during the episode did any violet cylinder move more than 8 mm sideways or tilt more than 15 degrees. Success is judged by the episode server from the physical state after you call robo done and the robots have held still for about 10 steps (0.5 s). "Released" means neither arm touches the object; "at rest" means it moves slower than 1 cm/s.

Goal fields in robo observe: tray_center (tray floor centre) and tray_inner_half_size (0.028 m; the tray has a 6 mm rim).

Controls. You drive the right arm only (the other arm stays parked). robo act DX DY DZ GRIP moves the arm's commanded gripper target by DX, DY, DZ times 2 cm per step along world x, y, z (each in [-1, 1], so 0.25 = 5 mm); GRIP 0 keeps the fingers as they are, any positive value closes them, and a negative value -f opens them to fraction f of full width (-1 fully open, -0.4 = 40 % open). One step is 50 ms of simulated time. Inverse kinematics turns the target into joint commands; the gripper always points straight down. robo move-to X Y Z and robo grip G drive the same arm (move-to stops within --tol metres of the point, default 0.01).

Observation. robo observe reports, for each arm you drive, hand_pos (the measured point between the fingertips), hand_target (where the arm is being driven; it can run up to 5 cm ahead of the hand, which then catches up), gripper_open (0 closed .. 1 fully open) and touching (task objects in contact with that arm); for every task object <name>_pos (the centre of the object's base, where it touches the surface below), <name>_tilt_deg (tilt from upright) and <name>_yaw_deg; plus the goal fields listed above. robo observe --image saves a picture from the front camera (--camera overhead looks straight down with +y at the top of the picture; --camera overview is a far corner view).

The step budget is 800 steps (40 s of simulated time).