Robo Use

farama-foundation/metaworld

Meta-World MT50 manipulation on a Sawyer arm.

bench eval run \
  -d farama-foundation/metaworld@0.1 \
  --registry https://robouse.ai/hub/registry.json \
  --agent oracle
About this dataset
Embodiment
Sawyer arm
Simulator
Meta-World 3.1.1
Requirements
CPU only · 1.1 GB image · bench >=0.7.4,<0.8
Licence
Task packages Apache-2.0; adapted from Meta-World by Farama Foundation, MIT
@misc{robouse_farama_foundation_metaworld_0_1,
  title        = {Robo Use farama-foundation/metaworld 0.1},
  author       = {{BenchFlow}},
  year         = {2026},
  howpublished = {\url{https://robouse.ai/hub/farama-foundation/metaworld/}},
  note         = {BenchFlow dataset, tag farama-foundation/metaworld-v0.1, commit 9e672aa1e7f0; adapted from Meta-World (Farama Foundation)}
}

50 tasks

Filters
Preview
Details

↑ ↓ move between rows · Enter shows details · O opens the task

Each task is a package in the task format, pinned to robohub commit 9e672aa1e7f0. The verifier and the reference solution are not published.

Full leaderboard

Agent harnesses

Rank#Model · harnessSolve rateSolvedTasks run
1Opus 5.5 · Claude Code 96% 48 of 50 50 of 50
2GPT-6 Astra · Codex 94% 47 of 50 50 of 50
3GLM-5.3 · Codex 84% 38 of 45 50 of 50 · 5 infra
4GLM-5.3 · mini-swe-agent 82% 41 of 50 50 of 50
5Kimi K3 · Claude Code 82% 41 of 50 50 of 50
6GLM-5.3 · Claude Code 80% 40 of 50 50 of 50
7Kimi K3 · mini-swe-agent 80% 40 of 50 50 of 50

VLA policies

Rank#Model · harnessSolve rateSolvedTasks run
Zero-shot probe (ranked on lifelong-robot-learning/libero), not ranked here
MolmoAct2-LIBERO · VLA 0% 0 of 5 5 of 50
GR00T N1.7-LIBERO · VLA 0% 0 of 3 3 of 50
pi0.5-LIBERO · VLA 0% 0 of 3 3 of 50

Per task

✓ solved ✗ not solved ! infrastructure failure · not run

TaskOpus 5.5 · Claude CodeGPT-6 Astra · CodexGLM-5.3 · CodexGLM-5.3 · mini-swe-agentKimi K3 · Claude CodeGLM-5.3 · Claude CodeKimi K3 · mini-swe-agentMolmoAct2-LIBERO · VLAGR00T N1.7-LIBERO · VLApi0.5-LIBERO · VLA
Pick up the round nut by its handle and place it over the pegmetaworld-assembly✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved···
Pick up the ball and dunk it into the basketball hoopmetaworld-basketball✓ Solved✓ Solved! Infrastructure failure✓ Solved✓ Solved✓ Solved✓ Solved···
Pick the cube out of the left bin and place it in the right binmetaworld-bin-picking✓ Solved✓ Solved! Infrastructure failure✓ Solved✓ Solved✓ Solved✓ Solved···
Pick up the box lid and place it on top of the boxmetaworld-box-close✓ Solved✓ Solved! Infrastructure failure✓ Solved✓ Solved✓ Solved✓ Solved···
Press the button on the wall panel by pushing it horizontallymetaworld-button-press✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Press the button down from abovemetaworld-button-press-topdown✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved··
Press the button down from above; a short wall stands in front of itmetaworld-button-press-topdown-wall✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Press the button horizontally; reach it past the short wall in front of itmetaworld-button-press-wall✓ Solved✓ Solved✓ Solved✗ Not solved✗ Not solved✓ Solved✓ Solved···
Press the button on the coffee machinemetaworld-coffee-button✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pull the mug out from under the coffee machine to the targetmetaworld-coffee-pull✓ Solved✓ Solved✓ Solved✗ Not solved✓ Solved✗ Not solved✓ Solved···
Push the mug under the coffee machine to the targetmetaworld-coffee-push✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Rotate the dial by pushing its handle aroundmetaworld-dial-turn✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✓ Solved···
Pick the nut off the peg by its handle and lift it clearmetaworld-disassemble✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✓ Solved···
Close the open door by pushing it shutmetaworld-door-close✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Lock the door by turning the lock knob downmetaworld-door-lock✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved···
Open the door by pulling its handlemetaworld-door-open✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved···
Unlock the door by turning the lock knobmetaworld-door-unlock✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✓ Solved✗ Not solved···
Push the drawer closedmetaworld-drawer-close✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pull the drawer open by its handlemetaworld-drawer-open✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved··
Turn the faucet handle to close itmetaworld-faucet-close✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Turn the faucet handle to open itmetaworld-faucet-open✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved···
Pick up the hammer and use it to hammer the nail into the wallmetaworld-hammer✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Push or carry the block into the hole in the table (the block must end within 5 cm of the hole)metaworld-hand-insert✓ Solved✓ Solved! Infrastructure failure✓ Solved✓ Solved✓ Solved✓ Solved···
Press the handle downmetaworld-handle-press✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Press the handle down, approaching from the sidemetaworld-handle-press-side✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pull the handle upmetaworld-handle-pull✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pull the handle up, approaching from the sidemetaworld-handle-pull-side✓ Solved✓ Solved✓ Solved✗ Not solved✓ Solved✓ Solved✓ Solved···
Pull the lever up to the goalmetaworld-lever-pull✗ Not solved✓ Solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved···
Pick up the peg and insert it sideways into the hole in the boxmetaworld-peg-insert-side✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pull the peg out sideways from the hole in the boxmetaworld-peg-unplug-side✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✓ Solved···
Pick the peg out of the hole and lift it to the goalmetaworld-pick-out-of-hole✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pick up the puck and place it at the goal (the goal is in the air)metaworld-pick-place✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved
Pick up the puck and place it at the goal on the other side of a wallmetaworld-pick-place-wall✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Slide the plate into the goal netmetaworld-plate-slide✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Slide the plate back out of the net to the goalmetaworld-plate-slide-back✓ Solved✗ Not solved✗ Not solved✓ Solved✗ Not solved✗ Not solved✗ Not solved···
Slide the plate sideways out of the net to the goalmetaworld-plate-slide-back-side✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved···
Slide the plate sideways into the goal netmetaworld-plate-slide-side✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Push the puck to the goal on the tablemetaworld-push✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved
Pull the puck back toward the robot to the goalmetaworld-push-back✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Push the puck to the goal, around the wall between themmetaworld-push-wall✓ Solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved✗ Not solved···
Move the gripper to the goal positionmetaworld-reach✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✗ Not solved✗ Not solved✗ Not solved
Move the gripper to the goal position, reaching over the wallmetaworld-reach-wall✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pick up the block and place it on the shelf at the goalmetaworld-shelf-place✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Push the ball into the goalmetaworld-soccer✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Pick up the stick and use it to pull the thermos to the goalmetaworld-stick-pull✗ Not solved✗ Not solved! Infrastructure failure✗ Not solved✗ Not solved✗ Not solved✗ Not solved···
Pick up the stick and use it to push the thermos to the goalmetaworld-stick-push✓ Solved✓ Solved✗ Not solved✓ Solved✓ Solved✓ Solved✓ Solved···
Sweep the puck off the edge of the tablemetaworld-sweep✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Sweep the puck into the hole in the tablemetaworld-sweep-into✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Slide the window closed by pushing its handle sidewaysmetaworld-window-close✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···
Slide the window open by pushing its handle sidewaysmetaworld-window-open✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved✓ Solved···