Cover the blocks left to right, then uncover them red, green, blue

RoboDojo task cover_blocks, capability dimension memory, evaluation layout 0.

RoboDojo's instruction: "Cover the blocks from left to right, remember their colors, then uncover them in the order: red, green, and blue."

You control RoboDojo's simulated ARX X5 bimanual platform in NVIDIA Isaac Sim 5.1: two 6-joint ARX X5 arms, each with a two-finger parallel gripper (about 9 cm fully open), mounted side by side at the near edge of a table, 0.6 m apart. World frame (metres): +x points to the robots' right, +y away from the robots across the table, +z up; the left arm's base is at about (-0.3, -0.45) and the right arm's at (0.3, -0.45), both 0.765 m above the floor, level with the table top (z = 0.765). Objects on the left half of the table (x < 0) are easiest for the left arm, those on the right half for the right arm.

Goal. Three coloured blocks (labels red, green, blue) stand in a row, with three upside-down cups in front of them. Cover the blocks with the cups one at a time from left to right (smallest x first), then lift the cups off again in the order red, green, blue, and finally move both arms back to their starting pose. The cups must stay upside down, and the blocks must stay where they are.

Success. RoboDojo's stages, in order (a block counts as covered when a cup's footprint contains the block's footprint, the block's top is below the cup's top, and the cup's bottom is less than 3 cm above the block's bottom; every cup must be upside down within 10 degrees at each stage): only the leftmost block covered; the left and middle blocks covered; all three covered; red uncovered with green and blue still covered; red and green uncovered with blue covered; all three uncovered and both grippers are back at their starting pose (within 15 cm along each axis and 20 degrees). Moving any block more than 5 cm from where it started fails the episode at once. This is RoboDojo's own run_reward() for cover_blocks, checked after every step.

Controls. robo act takes 16 numbers, RoboDojo's own end-effector action: for the left arm, then the right arm, the absolute target pose of the end-effector link (link6) as x y z and a quaternion qw qx qy qz, then the gripper opening (0 closed, 1 fully open). RoboDojo solves each target with its cuRobo inverse kinematics; a target the arm cannot reach leaves that arm where it is (the step result's ik field says Fail). One step is one RoboDojo policy step: 40 ms, 10 physics steps of 4 ms, with RoboDojo's interpolation. The skills are easier to use; each runs as a series of such steps and every step counts against the budget:

• robo skill move ARM X Y Z [PITCH] [YAW] [SPEED]: move the grasp point of one gripper (between the fingertips, 14.5 cm along the fingers from link6) in a straight line to (X, Y, Z); the other arm holds still. PITCH is how far below horizontal the fingers point (default 90: straight down) and YAW is the compass heading of the finger direction (default 90); with the fingers pointing down, the fingers close along the horizontal direction (-sin YAW, cos YAW, 0), so YAW = 90 closes them along x and YAW = 0 along y. SPEED is the most it moves per step (default 0.012 m).
• robo skill grip ARM OPEN [STEPS]: set one gripper's opening (0 closed .. 1 open) and hold for STEPS steps (default 12).
• robo skill home [ARM]: move one arm or both (default) back to the starting pose.
• robo skill wait [STEPS]: hold both arms still.

ARM is left or right. A skill call returns after at most 90 s of wall time; if it has not finished, its result says stopped and the same call continues from where the arm is.

Observation. robo observe reports, per arm under robot: ee_pos / ee_quat (the end-effector link pose), ee_axes (its x axis is the finger direction, its y axis the closing direction), tcp_pos (the grasp point), gripper_open (0..1), joints, and home_pos / home_quat (the starting pose). Under objects, every task object with its RoboDojo label, category, description, centre pos, quat (w, x, y, z), up_axis (the object's local z axis in the world), yaw_deg, its axis-aligned bounding box bbox_min / bbox_max and its size in its own frame; articulated objects (buttons, lids) also list their joints. instruction is RoboDojo's instruction for this episode, and steps / step_lim / stages_left show progress (stages_left counts RoboDojo's remaining success stages, which must be passed in order). robo observe --image saves RoboDojo's head camera (cam_head); --camera cam_left_wrist and --camera cam_right_wrist give the wrist cameras.

The step budget is RoboDojo's step_lim for this task: 800 steps (32 s of simulated time).