farama-foundation/gymnasium-robotics

Fetch reach, push and pick-and-place, and PointMaze.

bench eval run \
  -d farama-foundation/gymnasium-robotics@0.1 \
  --registry https://robouse.ai/hub/registry.json \
  --agent oracle

About this dataset

Embodiment: Fetch arm, Point mass
Simulator: Gymnasium-Robotics 1.4.2
Requirements: CPU only · 1.1 GB image · bench >=0.7.4,<0.8
Licence: Task packages Apache-2.0; adapted from Gymnasium-Robotics by Farama Foundation, MIT

@misc{robouse_farama_foundation_gymnasium_robotics_0_1,
  title        = {Robo Use farama-foundation/gymnasium-robotics 0.1},
  author       = {{BenchFlow}},
  year         = {2026},
  howpublished = {\url{https://robouse.ai/hub/farama-foundation/gymnasium-robotics/}},
  note         = {BenchFlow dataset, tag farama-foundation/gymnasium-robotics-v0.1, commit 9e672aa1e7f0; adapted from Gymnasium-Robotics (Farama Foundation)}
}

Filter tasks

12 tasks

Sort

Filters

Embodiment

Preview

Details

Pick up the block and place it at the goal position seed 0gymrobotics-fetch-pick-place-s0Pick up the block and place it at the goal position. The goal may be on the table or in the air; if it is in the air, hold the block there.

Fetch arm

300

7 of 7

Pick up the block and place it at the goal position. The goal may be on the table or in the air; if it is in the air, hold the block there.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	54	51 s	Trial
Kimi K3 · Claude Code	Solved	52	23 s	Trial
Kimi K3 · mini-swe-agent	Solved	104	4 min	Trial
GLM-5.3 · mini-swe-agent	Solved	89	81 s	Trial
GLM-5.3 · Claude Code	Solved	97	84 s	Trial
Opus 5.5 · Claude Code	Solved	43	37 s	Trial
GLM-5.3 · Codex	Solved	66	46 s	Trial

Open the task page Compare models

Pick up the block and place it at the goal position seed 1gymrobotics-fetch-pick-place-s1Pick up the block and place it at the goal position. The goal may be on the table or in the air; if it is in the air, hold the block there.

Fetch arm

300

7 of 7

Pick up the block and place it at the goal position. The goal may be on the table or in the air; if it is in the air, hold the block there.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	46	52 s	Trial
Kimi K3 · Claude Code	Solved	74	33 s	Trial
Kimi K3 · mini-swe-agent	Solved	234	2 min	Trial
GLM-5.3 · mini-swe-agent	Solved	161	2 min	Trial
GLM-5.3 · Claude Code	Solved	57	70 s	Trial
Opus 5.5 · Claude Code	Solved	47	36 s	Trial
GLM-5.3 · Codex	Solved	59	49 s	Trial

Open the task page Compare models

Push the block across the table so it rests on the goal position seed 0gymrobotics-fetch-push-s0

Fetch arm

300

6 of 7

Push the block across the table so it rests on the goal position.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	46	60 s	Trial
Kimi K3 · Claude Code	Solved	90	2 min	Trial
Kimi K3 · mini-swe-agent	Solved	55	4 min	Trial
GLM-5.3 · mini-swe-agent	Solved	104	3 min	Trial
GLM-5.3 · Claude Code	Solved	89	69 s	Trial
Opus 5.5 · Claude Code	Solved	41	38 s	Trial
GLM-5.3 · Codex	Not solved	300	4 min	Trial

Open the task page Compare models

Push the block across the table so it rests on the goal position seed 1gymrobotics-fetch-push-s1

Fetch arm

300

6 of 7

Push the block across the table so it rests on the goal position.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	33	56 s	Trial
Kimi K3 · mini-swe-agent	Solved	43	2 min	Trial
GLM-5.3 · mini-swe-agent	Solved	36	40 s	Trial
GLM-5.3 · Claude Code	Solved	22	3 min	Trial
Opus 5.5 · Claude Code	Solved	29	36 s	Trial
GLM-5.3 · Codex	Solved	30	29 s	Trial
Kimi K3 · Claude Code	Not solved	300	84 s	Trial

Open the task page Compare models

Move the gripper to the goal position and hold it there seed 0gymrobotics-fetch-reach-s0

Fetch arm

300

7 of 7

Move the gripper to the goal position and hold it there.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	32	20 s	Trial
Kimi K3 · Claude Code	Solved	42	26 s	Trial
Kimi K3 · mini-swe-agent	Solved	48	25 s	Trial
GLM-5.3 · mini-swe-agent	Solved	52	21 s	Trial
GLM-5.3 · Claude Code	Solved	38	33 s	Trial
Opus 5.5 · Claude Code	Solved	27	15 s	Trial
GLM-5.3 · Codex	Solved	32	20 s	Trial

Open the task page Compare models

Move the gripper to the goal position and hold it there seed 1gymrobotics-fetch-reach-s1

Fetch arm

300

7 of 7

Move the gripper to the goal position and hold it there.

Fetch arm300 stepsjudged on the final state

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	18	24 s	Trial
Kimi K3 · Claude Code	Solved	18	12 s	Trial
Kimi K3 · mini-swe-agent	Solved	18	18 s	Trial
GLM-5.3 · mini-swe-agent	Solved	18	14 s	Trial
GLM-5.3 · Claude Code	Solved	18	15 s	Trial
Opus 5.5 · Claude Code	Solved	13	13 s	Trial
GLM-5.3 · Codex	Solved	20	18 s	Trial

Open the task page Compare models

Roll the ball through the maze to the goal large seed 1gymrobotics-pointmaze-large-s1

Point mass

1,500

6 of 7

Roll the ball through the maze to the goal.

Point mass1,500 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	364	50 s	Trial
Kimi K3 · mini-swe-agent	Solved	699	3 min	Trial
GLM-5.3 · mini-swe-agent	Solved	702	43 s	Trial
GLM-5.3 · Claude Code	Solved	671	2 min	Trial
Opus 5.5 · Claude Code	Solved	548	33 s	Trial
GLM-5.3 · Codex	Solved	609	2 min	Trial
Kimi K3 · Claude Code	Not solved	1,500	2 min	Trial

Open the task page Compare models

Roll the ball through the maze to the goal large seed 2gymrobotics-pointmaze-large-s2

Point mass

1,500

5 of 7

Roll the ball through the maze to the goal.

Point mass1,500 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	370	70 s	Trial
Kimi K3 · mini-swe-agent	Solved	950	14 min	Trial
GLM-5.3 · Claude Code	Solved	678	4 min	Trial
Opus 5.5 · Claude Code	Solved	802	46 s	Trial
GLM-5.3 · Codex	Solved	545	2 min	Trial
Kimi K3 · Claude Code	Not solved	1,500	72 s	Trial
GLM-5.3 · mini-swe-agent	Not solved	1,500	8 min	Trial

Open the task page Compare models

Roll the ball through the maze to the goal medium seed 3gymrobotics-pointmaze-medium-s3

Point mass

1,000

6 of 7

Roll the ball through the maze to the goal.

Point mass1,000 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	224	54 s	Trial
Kimi K3 · Claude Code	Solved	428	30 s	Trial
GLM-5.3 · mini-swe-agent	Solved	699	58 s	Trial
GLM-5.3 · Claude Code	Solved	494	67 s	Trial
Opus 5.5 · Claude Code	Solved	441	32 s	Trial
GLM-5.3 · Codex	Solved	719	78 s	Trial
Kimi K3 · mini-swe-agent	Not solved	1,000	12 min	Trial

Open the task page Compare models

Roll the ball through the maze to the goal medium seed 5gymrobotics-pointmaze-medium-s5

Point mass

1,000

6 of 6

Roll the ball through the maze to the goal.

Point mass1,000 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	297	73 s	Trial
Kimi K3 · Claude Code	Solved	611	28 s	Trial
Kimi K3 · mini-swe-agent	Solved	467	90 s	Trial
GLM-5.3 · mini-swe-agent	Solved	806	3 min	Trial
GLM-5.3 · Claude Code	Solved	413	90 s	Trial
Opus 5.5 · Claude Code	Solved	632	38 s	Trial
GLM-5.3 · Codex	Infra failure	244	89 s	Trial

Open the task page Compare models

Roll the ball through the maze to the goal umaze seed 11gymrobotics-pointmaze-umaze-s11

Point mass

600

7 of 7

Roll the ball through the maze to the goal.

Point mass600 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	151	35 s	Trial
Kimi K3 · Claude Code	Solved	218	41 s	Trial
Kimi K3 · mini-swe-agent	Solved	280	60 s	Trial
GLM-5.3 · mini-swe-agent	Solved	315	57 s	Trial
GLM-5.3 · Claude Code	Solved	152	58 s	Trial
Opus 5.5 · Claude Code	Solved	119	25 s	Trial
GLM-5.3 · Codex	Solved	193	44 s	Trial

Open the task page Compare models

Roll the ball through the maze to the goal umaze seed 3gymrobotics-pointmaze-umaze-s3

Point mass

600

7 of 7

Roll the ball through the maze to the goal.

Point mass600 stepsends when first reached

Model	Result	Steps	Time	Trial
GPT-6 Astra · Codex	Solved	123	31 s	Trial
Kimi K3 · Claude Code	Solved	104	27 s	Trial
Kimi K3 · mini-swe-agent	Solved	193	54 s	Trial
GLM-5.3 · mini-swe-agent	Solved	127	79 s	Trial
GLM-5.3 · Claude Code	Solved	162	3 min	Trial
Opus 5.5 · Claude Code	Solved	122	28 s	Trial
GLM-5.3 · Codex	Solved	416	61 s	Trial

Open the task page Compare models

↑ ↓ move between rows · Enter shows details · O opens the task

Each task is a package in the task format, pinned to robohub commit 9e672aa1e7f0. The verifier and the reference solution are not published.

Full leaderboard

Agent harnesses

Rank#	Model · harness	Solve rate	Solved	Tasks run
1	GLM-5.3 · Claude Code	100%	12 of 12	12 of 12
2	GPT-6 Astra · Codex	100%	12 of 12	12 of 12
3	Opus 5.5 · Claude Code	100%	12 of 12	12 of 12
4	GLM-5.3 · mini-swe-agent	92%	11 of 12	12 of 12
5	Kimi K3 · mini-swe-agent	92%	11 of 12	12 of 12
6	GLM-5.3 · Codex	91%	10 of 11	12 of 12 · 1 infra
7	Kimi K3 · Claude Code	75%	9 of 12	12 of 12

Per task

✓ solved ✗ not solved ! infrastructure failure · not run

Task	GLM-5.3 · Claude Code	GPT-6 Astra · Codex	Opus 5.5 · Claude Code	GLM-5.3 · mini-swe-agent	Kimi K3 · mini-swe-agent	GLM-5.3 · Codex	Kimi K3 · Claude Code
Pick up the block and place it at the goal position`gymrobotics-fetch-pick-place-s0`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved
Pick up the block and place it at the goal position`gymrobotics-fetch-pick-place-s1`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved
Push the block across the table so it rests on the goal position`gymrobotics-fetch-push-s0`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✗ Not solved	✓ Solved
Push the block across the table so it rests on the goal position`gymrobotics-fetch-push-s1`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✗ Not solved
Move the gripper to the goal position and hold it there`gymrobotics-fetch-reach-s0`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved
Move the gripper to the goal position and hold it there`gymrobotics-fetch-reach-s1`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-large-s1`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✗ Not solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-large-s2`	✓ Solved	✓ Solved	✓ Solved	✗ Not solved	✓ Solved	✓ Solved	✗ Not solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-medium-s3`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✗ Not solved	✓ Solved	✓ Solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-medium-s5`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	! Infrastructure failure	✓ Solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-umaze-s11`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved
Roll the ball through the maze to the goal`gymrobotics-pointmaze-umaze-s3`	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved	✓ Solved