Robo Use

Overview

Robo Use turns robot benchmarks into tasks that an agent harness, such as Claude Code or Codex, solves by driving a simulated robot from its shell with the robo command. An episode server runs the simulator, enforces the step budget, records a video and scores the final physical state.

pip install 'robouse[sim]'
export CLAUDE_CODE_OAUTH_TOKEN=...
robouse run \
  --task arc-gravity \
  --harness claude-code \
  --out runs/quickstart
Agent harness such as Claude Code or Codex prompt: the task.md body in a fresh workspace: observations/ $ robo act 0 1 0 -1 sandboxed: Seatbelt on macOS, bubblewrap on Linux JSON request Unix socket Episode server the runner starts one per trial simulator (MuJoCo) step and time budgets video and trace judges success trusted: the agent can't reach it the runner saves its output records the episode Trial folder OUT/<task>__<harness>__<id>/ agent/trajectory.json episode/result.json artifacts/recording.mp4 agent/stdout.jsonl episode/trace.jsonl verifier/reward.txt Agent harness such as Claude Code or Codex prompt: the task.md body in a fresh workspace: observations/ $ robo act 0 1 0 -1 sandboxed: Seatbelt on macOS, bubblewrap on Linux JSON request Unix socket Episode server the runner starts one per trial simulator (MuJoCo) step and time budgets video and trace judges success trusted: the agent can't reach it records the episode the runner saves its output Trial folder OUT/<task>__<harness>__<id>/ agent/trajectory.json agent/stdout.jsonl episode/result.json episode/trace.jsonl artifacts/recording.mp4 verifier/reward.txt
One trial: the harness sends robo requests over a Unix socket to the episode server, which owns the simulator; the trial folder collects the agent's output, the episode record and the reward.

Every task is a BenchFlow task folder (task.md) with a reference solution and a no-op control (Suites lists their scores). Every trial is a folder in BenchFlow's trial layout: video, trajectory and reward. Published datasets of these tasks run with BenchFlow's bench eval run -d.

Tasks come from Meta-World, LIBERO, robosuite, Gymnasium-Robotics, dm_control, MyoSuite, HumanoidBench, ManiSkill3, RoboCasa, DexJoCo, MetaDrive, Robocurve's Inspect Robots and MuJoCo Menagerie robots, plus Robo Use's own suites. Results: Runs.