Scenario reference
The observations, actions and success criteria of each scenario.
This page was derived from the scenario definitions and task classes in resources/scenarios: the observation keys,
action limits and success criteria are what your policy sees and must return in each scenario.
These are simplified arm models
Verify scenarios use small arm and gripper models built for fast, repeatable simulation, so the action vector is 3 or 6 actuator controls, not the full joint count of the real robot named in each section.
Messy Room
messy_room · Easy · robot: Franka Panda
Robot arm must reach and touch the red target cube on a table while ignoring the blue distractor sphere.
Observation keys: joint_positions (3), end_effector_pos (3), target_pos (3), distractor_pos (3)
Action: a vector of 3 controls with limits ±10, ±10, ±10 (in order).
Success: the target object is touched.
Navigation
navigation · Easy · robot: Franka Panda
Robot arm must navigate its end-effector to reach a target marker position in 3D space.
Observation keys: joint_positions (3), joint_velocities (3), end_effector_pos (3), target_pos (3), distance_to_target, target_reached
Action: a vector of 3 controls with limits ±10, ±10, ±10 (in order).
Success: the end effector reaches the target.
Object Sorting
object_sorting · Easy · robot: Franka Panda
Robot arm must sort colored cubes into their matching colored bins - red cubes to red bin, blue cubes to blue bin.
Observation keys: joint_positions (3), end_effector_pos (3), red_bin_pos (3), blue_bin_pos (3), red_cube_positions (2x3), blue_cube_positions (2x3)
Action: a vector of 3 controls with limits ±10, ±10, ±10 (in order).
Success: an object ends up in its zone.
Obstacle Avoidance
obstacle_avoidance · Hard · robot: Franka Panda
Robot arm must navigate its end-effector to a target while avoiding multiple obstacles in a warehouse environment.
Observation keys: joint_positions (3), end_effector_pos (3), target_pos (3), distance_to_target, obstacle_positions (3x3), collision_detected, target_reached
Action: a vector of 3 controls with limits ±10, ±10, ±10 (in order).
Success: the end effector reaches the target; no contact with the obstacles.
Mug Placement
mug_placement · Easy · robot: KUKA iiwa
Robot arm must pick up a mug and place it upright on a designated coaster spot in the kitchen.
Observation keys: joint_positions (3), end_effector_pos (3), mug_pos (3), coaster_pos (3), mug_to_coaster, ee_to_mug
Action: a vector of 3 controls with limits ±10, ±10, ±10 (in order).
Success: an object ends up in its zone.
Kitchen Pickup
kitchen_pickup · Medium · robot: UR5e
Robot arm must pick up a mug from the counter and place it on a plate in a kitchen environment.
Observation keys: joint_positions (4), gripper_opening, gripper_pos (3), mug_pos (3), plate_pos (3), apple_pos (3), mug_to_plate_distance
Action: a vector of 6 controls with limits ±5, ±5, ±5, ±3, ±2, ±2 (in order).
Success: an object ends up in its zone; the video grader judges the task complete.
Table Clearing
table_clearing · Medium · robot: UR5e
Robot arm must clear all objects from the table by placing them into a collection bin.
Observation keys: joint_positions (4), gripper_opening, gripper_pos (3), apple_pos (3), mug_pos (3), plate_pos (3), bin_pos (3)
Action: a vector of 6 controls with limits ±5, ±5, ±5, ±3, ±2, ±2 (in order).
Success: an object ends up in its zone.
Warehouse Retrieval
warehouse_retrieval · Medium · robot: Unitree Go2
Mobile robot must navigate to a warehouse shelf, pick up a cardboard box, and bring it to the staging area.
Observation keys: robot_pos (3), box_pos (3), staging_pos (3), robot_to_box, box_to_staging, arm_joint
Action: a vector of 3 controls with limits ±10, ±10, ±5 (in order).
Success: an object ends up in its zone.
Dual Arm Handover
dual_arm_handover · Medium · robot: ALOHA (dual arm)
Dual-arm ALOHA robot must pick up a cube with the left arm and hand it over to the right arm.
Observation keys: left_ee_pos (3), right_ee_pos (3), cube_pos (3), target_pos (3), cube_to_target, left_joints (3), right_joints (3)
Action: a vector of 6 controls with limits ±10 for all six (in order).
Success: an object ends up in its zone; the video grader judges the task complete.
Block Stacking
block_stacking · Hard · robot: Franka Panda
Robot arm must stack colored blocks in the correct order: red on bottom, blue in middle, green on top.
Observation keys: joint_positions (4), gripper_opening, gripper_pos (3), platform_pos (3), block_positions, correct_order (3)
Action: a vector of 6 controls with limits ±5, ±5, ±5, ±3, ±2, ±2 (in order).
Success: an object ends up in its zone; blocks are stacked in order; the video grader judges the task complete.
Precision Placement
precision_placement · Hard · robot: Franka Panda
Robot arm must pick up colored cylinders and place them precisely at their matching colored target markers.
Observation keys: joint_positions (4), gripper_opening, gripper_pos (3), object_positions, target_positions, placement_errors
Action: a vector of 6 controls with limits ±5, ±5, ±5, ±3, ±2, ±2 (in order).
Success: an object ends up in its zone; the video grader judges the task complete.