Insert the peg into the side hole (ManiSkill3 PegInsertionSide-v1, Franka Panda, CPU physics)

A Franka Emika Panda arm (7 joints, two-finger parallel gripper that opens to 8 cm) stands at a wooden table in ManiSkill3's PegInsertionSide-v1 task, simulated with SAPIEN 3 (PhysX). The robot's base is at x = -0.615, y = 0 and faces +x, so +x points away from the robot and +y to its left. The table top is the plane z = 0.

Goal: Pick up the peg (a square bar, orange head end and white tail end) and insert its orange head end sideways into the hole in the box, at least halfway in.

Success: ManiSkill's own PegInsertionSide check: the peg's head is inside the hole, no more than 1.5 cm short of the hole centre along the hole axis and within the hole's radius of the axis sideways. The hole has only 3 mm of clearance around the peg. Success is judged by the episode server after you call robo done and the robot has held still for about 10 steps, so the result must last.

Task fields in robo observe: peg_pos (centre), peg_quat, peg_yaw_deg, peg_axis (unit vector from the tail toward the head), peg_head_pos (centre of the head end face), peg_half_length, peg_radius (half the peg's width), hole_pos (centre of the hole, halfway through the box), hole_axis (unit vector along which the peg goes in; the hole's entrance is at hole_pos - peg_half_length x hole_axis), hole_radius and grasped.

Controls. The action has 7 numbers, not 4: robo act DX DY DZ DROLL DPITCH DYAW GRIP, each in [-1, 1], in the WORLD frame. DX/DY/DZ move the gripper's commanded position by 2 cm per unit per step. DROLL/DPITCH/DYAW turn its commanded orientation by 0.1 rad (5.7 degrees) per unit per step about the world x, y and z axes, around the point between the fingertips; leave them at 0 unless you need to turn the hand (for example DYAW to line the fingers up with an object's faces). GRIP: any positive value closes the fingers, a negative value -f opens them to fraction f (-1 fully open), 0 keeps them as they are (they take about 5 steps to move). One step is 50 ms (ManiSkill's 20 Hz control rate). The arm follows the commanded pose through inverse kinematics and ManiSkill's joint position controller; the commanded position stays within 5 cm of the measured hand (and the orientation within 0.35 rad), so pushing against something stalls instead of winding up. Some orientations are out of the arm's reach (the last wrist joint turns about 166 degrees each way): turning the fingers by 90 or 180 degrees about the vertical gives the same grasp on a cube, so prefer the smaller turn. robo move-to X Y Z [--grip G] and robo grip G also work; they move only the position and keep the hand's orientation, and move-to stops within 1 cm of the point unless you pass a tighter --tol (e.g. --tol 0.002).

Observation. robo observe reports hand_pos (the point between the fingertips, metres), hand_quat (hand orientation quaternion w, x, y, z), hand_down_axis (unit vector the fingers point along; [0, 0, -1] is straight down), finger_axis (unit vector along which the fingers close), hand_target_pos and hand_target_quat (the commanded pose the arm is moving to), gripper_open (0 closed .. 1 fully open) and gripper_width (distance between the fingers, metres), plus the task fields below. Every <object>_quat is (w, x, y, z); <object>_yaw_deg is the object's heading about the vertical (its local x axis, degrees from +x). grasped, where present, is ManiSkill's own grasp test (both fingers pressing on the object). robo observe --image saves a 384x384 picture from the render_camera view; --camera base_camera, --camera hand_camera (on the wrist) gives other views.

The step budget is 600 steps (30 s of simulated time).