Robo Use

Pick up the peg and insert its orange head into the hole in the side of the box, at least half-way

Pick up the peg and insert its orange head into the hole in the side of the box, at least half-way; the hole leaves only 1 mm of clearance on each side

ms-peg-insertion-side-1mm1mm variantgpu-maniskillFranka PandaManipulationhard

Reference solution, 21 of 40 steps.

Instruction

Pick up the peg lying on the table and insert it into the hole in the side of the box. The peg is a square bar: its head half is orange, its tail half white. Insert the orange head first: the task is solved when the peg's head end is inside the hole and at least half-way through the box's depth (the head end within 15 mm of the box centre along the hole or past it), and inside the hole's cross-section.

robo observe gives peg_pose (the peg's centre; its +x axis points from the tail to the head), peg_half_size (half length, half width, half height in metres), box_hole_pose (the centre of the hole at the middle of the box; its +x axis is the insertion direction, so the peg enters from the hole frame's -x side) and box_hole_radius (the half width of the square hole).

The hole leaves only 1 mm of clearance on each side of the peg.

The step budget is 40 steps: every robo act step and every robo skill call counts as one.

Scene and frames

World frame: x points away from the robot across the table, y to the robot's left, z up; the table top is z = 0 and the robot base sits at (-0.615, 0, 0). Positions are in metres. Poses in robo observe give pos, quat_wxyz (a unit quaternion w, x, y, z) and rpy_deg (roll, pitch, yaw in degrees: R = Rz(yaw) Ry(pitch) Rx(roll)).

The gripper's tool point (TCP) sits between the fingertips. In the TCP frame, +z points out between the fingers (the approach direction) and +y is the axis along which the fingers close. Pointing straight down with the fingers closing along world y is roll 180, pitch 0, yaw 0 (quaternion 0 1 0 0); turning that about the vertical is a change of yaw.

How the robot is controlled and scored

You are controlling a simulated robot. Read the task below, then solve it by running the robo command in your shell (start with robo info and robo observe). Keep going until the task is done, then call robo done once. Do not stop to ask questions; there is no human to answer.

How to control the robot

You are the robot's policy. You act only through the robo command in your shell; you cannot read or change the simulator, the scoring, or other files to succeed. The episode server checks the task's physical success condition on the final simulated state itself.

robo info                                      # action space, skills, cameras, step budget
robo observe                                   # TCP pose, joint angles, gripper, and the task's object state
robo observe --image [--camera C]              # also saves a ray-traced image (cameras: scene, front, top, wrist) and prints its path
robo skill move X Y Z                          # motion-planned move of the TCP to (X, Y, Z), keeping its orientation
robo skill move X Y Z ROLL PITCH YAW           # ... to a full pose, orientation in degrees
robo skill move X Y Z QW QX QY QZ              # ... orientation as a quaternion
robo skill move ... speed=0.3                  # any move, slowed down (speed 0.05-1, default 1)
robo skill gripper open|close                  # open or close the fingers
robo skill gripper -0.6                        # finger target in [-1, 1] (-1 closed, +1 open); a loose grip lets a held object pivot
robo skill wait [N]                            # hold still for N control steps (default 10)
robo act DX DY DZ DRX DRY DRZ GRIP [--repeat N]   # low-level: one 0.05 s control step (see below)
robo done "short summary"                      # end the episode and ask for scoring
robo give-up "reason"                          # end the episode without claiming success
  • move plans a path for the arm itself (straight-line screw motion first, then a sampling planner) but does not avoid the objects on the table: moving through them pushes them. It reports the TCP position it reached and the remaining position and orientation error. Contact (a held object touching something) can stop the arm short of the target.
  • act is ManiSkill's end-effector delta-pose controller: DX, DY, DZ move the TCP by up to 0.1 m per step (value x 0.1 m), DRX, DRY, DRZ rotate it by up to 0.1 rad per step about world-aligned axes, and GRIP is the absolute finger target (+1 open, -1 closed). Values are clipped to [-1, 1].
  • Success is judged after you call robo done, with the robot holding still for 10 control steps: the condition must hold at that moment. robo give-up, running out of steps or time, or stopping without robo done all score 0. Call robo done exactly once.

Run this task

robouse run \
  --task tasks/gpu-maniskill/ms-peg-insertion-side-1mm \
  --harness oracle \
  --out runs/ms-peg-insertion-side-1mm

Pinned to robohub commit e472b1a1e041. The verifier and the reference solution are not published.

No trials yet.