arise-initiative/robosuite
Tasks from robosuite 1.5 on five arm models.
bench eval run \
-d arise-initiative/robosuite@0.1 \
--registry https://robouse.ai/hub/registry.json \
--agent oracleAbout this dataset
- Embodiment
- Franka Panda, KUKA IIWA, Kinova Gen3, UR5e, Sawyer arm
- Simulator
- robosuite 1.5.2
- Requirements
- CPU only · 2.5 GB image ·
bench>=0.7.4,<0.8 - Licence
- Task packages Apache-2.0; adapted from robosuite by ARISE Initiative, MIT
@misc{robouse_arise_initiative_robosuite_0_1,
title = {Robo Use arise-initiative/robosuite 0.1},
author = {{BenchFlow}},
year = {2026},
howpublished = {\url{https://robouse.ai/hub/arise-initiative/robosuite/}},
note = {BenchFlow dataset, tag arise-initiative/robosuite-v0.1, commit 9e672aa1e7f0; adapted from robosuite (ARISE Initiative)}
}21 tasks
Embodiment
Size
Step budget
robosuite-door-iiwaOpen the door. It is latched: first turn its lever handle far enough to release the latch, then swing the door open toward the robot. robosuite's…Open the door. It is latched: first turn its lever handle far enough to release the latch, then swing the door open toward the robot. robosuite's Door check: the hinge angle hinge_qpos is above 0.3 rad (the hinge stops at 0.4 rad). The door must still be open when the robot stops after robo done.
KUKA IIWA400 stepsjudged on the final state
robosuite-door-pandaOpen the door. It is latched: first turn its lever handle far enough to release the latch, then swing the door open toward the robot. robosuite's…Open the door. It is latched: first turn its lever handle far enough to release the latch, then swing the door open toward the robot. robosuite's Door check: the hinge angle hinge_qpos is above 0.3 rad (the hinge stops at 0.4 rad). The door must still be open when the robot stops after robo done.
Franka Panda400 stepsjudged on the final state
| Model | Result | Steps | Time | Trial |
|---|---|---|---|---|
| GPT-6 Astra · Codex | Solved | 227 | 2 min | Trial |
| Kimi K3 · Claude Code | Solved | 148 | 7 min | Trial |
| GLM-5.3 · Claude Code | Solved | 151 | 2 min | Trial |
| GLM-5.3 · Codex | Solved | 217 | 71 s | Trial |
| Kimi K3 · mini-swe-agent | Not solved | 400 | 13 min | Trial |
| GLM-5.3 · mini-swe-agent | Not solved | 400 | 10 min | Trial |
| pi0.5-LIBERO · VLA | Not solved | 399 | 32 s | Trial |
| GR00T N1.7-LIBERO · VLA | Not solved | 399 | 32 s | Trial |
robosuite-lift-iiwaPick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top…Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top (robosuite's Lift check); the episode then ends by itself.
KUKA IIWA400 stepsends when first reached
robosuite-lift-kinova3Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top…Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top (robosuite's Lift check); the episode then ends by itself.
Kinova Gen3400 stepsends when first reached
robosuite-lift-pandaPick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top…Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top (robosuite's Lift check); the episode then ends by itself.
Franka Panda400 stepsends when first reached
| Model | Result | Steps | Time | Trial |
|---|---|---|---|---|
| GPT-6 Astra · Codex | Solved | 66 | 39 s | Trial |
| Kimi K3 · Claude Code | Solved | 58 | 37 s | Trial |
| Kimi K3 · mini-swe-agent | Solved | 75 | 78 s | Trial |
| GLM-5.3 · mini-swe-agent | Solved | 83 | 78 s | Trial |
| GLM-5.3 · Claude Code | Solved | 65 | 2 min | Trial |
| pi0.5-LIBERO · VLA | Solved | 125 | 16 s | Trial |
| GR00T N1.7-LIBERO · VLA | Not solved | 399 | 31 s | Trial |
| GLM-5.3 · Codex | Infra failure | 54 | 31 s | Trial |
robosuite-lift-sawyerPick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top…Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top (robosuite's Lift check); the episode then ends by itself.
Sawyer arm400 stepsends when first reached
robosuite-lift-ur5ePick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top…Pick up the red cube from the table and lift it. The task is solved the moment the cube's centre is more than 4 cm above the table top (robosuite's Lift check); the episode then ends by itself.
UR5e400 stepsends when first reached
robosuite-nut-round-pandaPick up the round nut by its handle and fit it over the round peg so that it slides down around the peg to the table. robosuite's NutAssembly…Pick up the round nut by its handle and fit it over the round peg so that it slides down around the peg to the table. robosuite's NutAssembly check: the nut's centre is within 3 cm of the peg's axis in x and y, below table_height + 0.05, and the gripper is at least about 4 cm away from it.
Franka Panda500 stepsjudged on the final state
robosuite-nut-square-pandaPick up the square nut by its handle and fit it over the square peg so that it slides down around the peg to the table. robosuite's NutAssembly…Pick up the square nut by its handle and fit it over the square peg so that it slides down around the peg to the table. robosuite's NutAssembly check: the nut's centre is within 3 cm of the peg's axis in x and y, below table_height + 0.05, and the gripper is at least about 4 cm away from it. The square nut's hole only fits the square peg when the nut is turned to match it (the peg's faces are aligned with the x and y axes).
Franka Panda500 stepsjudged on the final state
robosuite-pickplace-bread-pandaPick up the loaf of bread from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the…Pick up the loaf of bread from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's left (bin 2, at +y). robosuite's PickPlace check: the bread's centre lies inside the target compartment (x inside target_bin_x_range, y inside target_bin_y_range, z between bin_floor_z and bin_floor_z + 0.1) and the gripper is at least about 4 cm away from it.
Franka Panda400 stepsjudged on the final state
robosuite-pickplace-can-kinova3Pick up the soda can from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's…Pick up the soda can from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's left (bin 2, at +y). robosuite's PickPlace check: the can's centre lies inside the target compartment (x inside target_bin_x_range, y inside target_bin_y_range, z between bin_floor_z and bin_floor_z + 0.1) and the gripper is at least about 4 cm away from it.
Kinova Gen3400 stepsjudged on the final state
robosuite-pickplace-can-pandaPick up the soda can from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's…Pick up the soda can from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's left (bin 2, at +y). robosuite's PickPlace check: the can's centre lies inside the target compartment (x inside target_bin_x_range, y inside target_bin_y_range, z between bin_floor_z and bin_floor_z + 0.1) and the gripper is at least about 4 cm away from it.
Franka Panda400 stepsjudged on the final state
| Model | Result | Steps | Time | Trial |
|---|---|---|---|---|
| GPT-6 Astra · Codex | Solved | 227 | 51 s | Trial |
| Kimi K3 · Claude Code | Solved | 216 | 32 s | Trial |
| GLM-5.3 · mini-swe-agent | Solved | 268 | 59 s | Trial |
| GLM-5.3 · Claude Code | Solved | 231 | 2 min | Trial |
| GLM-5.3 · Codex | Solved | 205 | 44 s | Trial |
| pi0.5-LIBERO · VLA | Not solved | 399 | 33 s | Trial |
| GR00T N1.7-LIBERO · VLA | Not solved | 399 | 33 s | Trial |
| Kimi K3 · mini-swe-agent | Infra failure | 0 | 5 min | Trial |
robosuite-pickplace-cereal-pandaPick up the cereal box from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's…Pick up the cereal box from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's left (bin 2, at +y). robosuite's PickPlace check: the cereal's centre lies inside the target compartment (x inside target_bin_x_range, y inside target_bin_y_range, z between bin_floor_z and bin_floor_z + 0.1) and the gripper is at least about 4 cm away from it.
Franka Panda400 stepsjudged on the final state
robosuite-pickplace-milk-pandaPick up the milk carton from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's…Pick up the milk carton from the bin on the robot's right (bin 1, at -y) and put it into its target compartment of the divided bin on the robot's left (bin 2, at +y). robosuite's PickPlace check: the milk's centre lies inside the target compartment (x inside target_bin_x_range, y inside target_bin_y_range, z between bin_floor_z and bin_floor_z + 0.1) and the gripper is at least about 4 cm away from it.
Franka Panda400 stepsjudged on the final state
robosuite-stack-pandaPut the red cube (cubeA, 4 cm) on top of the green cube (cubeB, 5 cm) and let go. robosuite's Stack check: the red cube is lifted off the table,…Put the red cube (cubeA, 4 cm) on top of the green cube (cubeB, 5 cm) and let go. robosuite's Stack check: the red cube is lifted off the table, touching the green cube, and no longer held by the gripper.
Franka Panda400 stepsjudged on the final state
| Model | Result | Steps | Time | Trial |
|---|---|---|---|---|
| GPT-6 Astra · Codex | Solved | 134 | 65 s | Trial |
| Kimi K3 · Claude Code | Solved | 136 | 36 s | Trial |
| Kimi K3 · mini-swe-agent | Solved | 194 | 4 min | Trial |
| GLM-5.3 · mini-swe-agent | Solved | 159 | 2 min | Trial |
| GLM-5.3 · Claude Code | Solved | 164 | 10 min | Trial |
| GLM-5.3 · Codex | Solved | 112 | 44 s | Trial |
| pi0.5-LIBERO · VLA | Not solved | 399 | 34 s | Trial |
| GR00T N1.7-LIBERO · VLA | Not solved | 399 | 32 s | Trial |
robosuite-stack-ur5ePut the red cube (cubeA, 4 cm) on top of the green cube (cubeB, 5 cm) and let go. robosuite's Stack check: the red cube is lifted off the table,…Put the red cube (cubeA, 4 cm) on top of the green cube (cubeB, 5 cm) and let go. robosuite's Stack check: the red cube is lifted off the table, touching the green cube, and no longer held by the gripper.
UR5e400 stepsjudged on the final state
robosuite-twoarm-handover-pandaThe hammer lies on the table on robot0's side. Robot0 picks it up and hands it to robot1. robosuite's TwoArmHandover check: robot1 grasps the…The hammer lies on the table on robot0's side. Robot0 picks it up and hands it to robot1. robosuite's TwoArmHandover check: robot1 grasps the hammer's handle, robot0 no longer touches the hammer, and the hammer is more than 10 cm above the table. This must still hold after robo done.
Franka Panda700 stepsjudged on the final state
robosuite-twoarm-lift-iiwaLift the pot together: robot0 holds handle0, robot1 holds handle1. robosuite's TwoArmLift check: the pot's bottom is more than 10 cm above the…Lift the pot together: robot0 holds handle0, robot1 holds handle1. robosuite's TwoArmLift check: the pot's bottom is more than 10 cm above the table. The episode ends by itself at that moment.
KUKA IIWA400 stepsends when first reached
robosuite-twoarm-lift-pandaLift the pot together: robot0 holds handle0, robot1 holds handle1. robosuite's TwoArmLift check: the pot's bottom is more than 10 cm above the…Lift the pot together: robot0 holds handle0, robot1 holds handle1. robosuite's TwoArmLift check: the pot's bottom is more than 10 cm above the table. The episode ends by itself at that moment.
Franka Panda400 stepsends when first reached
robosuite-twoarm-peginhole-pandaRobot0 holds a peg (a cylinder fixed to its hand) and robot1 holds a square plate with a hole (fixed to its hand). Bring them together so the peg…Robot0 holds a peg (a cylinder fixed to its hand) and robot1 holds a square plate with a hole (fixed to its hand). Bring them together so the peg passes through the hole. robosuite's TwoArmPegInHole check: the peg's axis is nearly parallel to the hole's axis (|cos| > 0.95), the peg's centre is within 6 cm of the hole's axis, and it sits between 12 cm in front of and 14 cm behind the hole. Both must still hold after robo done.
Franka Panda500 stepsjudged on the final state
robosuite-wipe-pandaWipe all the dirt (100 small markers along a line on the table) off the table with the wiping tool mounted on the arm. A marker counts as wiped…Wipe all the dirt (100 small markers along a line on the table) off the table with the wiping tool mounted on the arm. A marker counts as wiped when the tool's bottom face passes over it while pressed flat on the table. robosuite's Wipe check: every marker is wiped. The episode ends by itself when the last marker goes.
Franka Panda600 stepsends when first reached
↑ ↓ move between rows · Enter shows details · O opens the task
Each task is a package in the task format, pinned to robohub commit 9e672aa1e7f0. The verifier and the reference solution are not published.
Agent harnesses
| Rank# | Model · harness | Solve rate | Solved | Tasks run |
|---|---|---|---|---|
| 1 | GPT-6 Astra · Codex | 95% | 20 of 21 | 21 of 21 |
| 2 | Kimi K3 · mini-swe-agent | 93% | 14 of 15 | 21 of 21 · 6 infra |
| 3 | Kimi K3 · Claude Code | 90% | 19 of 21 | 21 of 21 |
| 4 | GLM-5.3 · Claude Code | 85% | 17 of 20 | 21 of 21 · 1 infra |
| 5 | GLM-5.3 · mini-swe-agent | 70% | 14 of 20 | 21 of 21 · 1 infra |
| 6 | GLM-5.3 · Codex | 60% | 6 of 10 | 21 of 21 · 11 infra |
VLA policies
| Rank# | Model · harness | Solve rate | Solved | Tasks run |
|---|---|---|---|---|
| Zero-shot probe (ranked on lifelong-robot-learning/libero), not ranked here | ||||
| pi0.5-LIBERO · VLA | 25% | 1 of 4 | 4 of 21 | |
| GR00T N1.7-LIBERO · VLA | 0% | 0 of 4 | 4 of 21 | |
Per task
✓ solved ✗ not solved ! infrastructure failure · not run