Tasks Overview#

RoboEval includes 8 task families with 28 total variations designed to test different aspects of bimanual manipulation.

Task Families#

Task Family	Variations	Description
Lift Pot	4	Grasp pot by handles and lift without tilting
Stack Books	3	Stack books on shelves requiring precise placement
Stack Books	4	Pick books from table and transfer between hands
Manipulation Tasks	4	Stack blocks requiring bimanual coordination
Manipulation Tasks	5	Transfer cube between robot hands (includes vertical handover)
Rotate Valve	3	Rotate valve wheel with continuous bimanual coordination
Pack Box	4	Pack objects into a box requiring spatial reasoning
Lift Tray	5	Lift tray with objects maintaining balance (includes drag-over variant)

Variation Types#

Most tasks have 4 standard variations:

Base - Static configuration: Objects appear in same position every episode
Position - Positional randomization: Object positions randomized within workspace
Orientation - Rotational randomization: Object orientations randomized (typically ±30° around Z-axis)
Position and Orientation - Full randomization: Both position and orientation randomized

Quick Reference#

# Lift Pot (4 variations)
from roboeval.envs.lift_pot import (
    LiftPot, LiftPotPosition,
    LiftPotOrientation, LiftPotPositionAndOrientation
)

# Stack Single Book Shelf (3 variations)
from roboeval.envs.stack_books import (
    StackSingleBookShelf, StackSingleBookShelfPosition,
    StackSingleBookShelfPositionAndOrientation
)

# Pick Single Book From Table (4 variations)
from roboeval.envs.stack_books import (
    PickSingleBookFromTable, PickSingleBookFromTablePosition,
    PickSingleBookFromTableOrientation, PickSingleBookFromTablePositionAndOrientation
)

# Stack Two Blocks (4 variations)
from roboeval.envs.manipulation import (
    StackTwoBlocks, StackTwoBlocksPosition,
    StackTwoBlocksOrientation, StackTwoBlocksPositionAndOrientation
)

# Cube Handover (5 variations)
from roboeval.envs.manipulation import (
    CubeHandover, CubeHandoverPosition, CubeHandoverOrientation,
    CubeHandoverPositionAndOrientation, VerticalCubeHandover
)

# Rotate Valve (3 variations)
from roboeval.envs.rotate_utility_objects import (
    RotateValve, RotateValvePosition, RotateValvePositionAndOrientation
)

# Pack Box (4 variations)
from roboeval.envs.pack_objects import (
    PackBox, PackBoxPosition, PackBoxOrientation, PackBoxPositionAndOrientation
)

# Lift Tray (5 variations)
from roboeval.envs.lift_tray import (
    LiftTray, LiftTrayPosition, LiftTrayOrientation,
    LiftTrayPositionAndOrientation, DragOverAndLiftTray
)

Task Progression Stages#

Each task defines progression stages for fine-grained evaluation:

Example: LiftPot

Stage 0: Initial state
Stage 1: Left gripper grasping pot
Stage 2: Right gripper grasping pot
Stage 3: Pot lifted 0.1m above table
Stage 4: Success (pot stable and level)

Access stages during execution:

obs, reward, terminated, truncated, info = env.step(action)
current_stage = info['metrics']['stage']

Metrics#

All tasks provide rich metrics:

Success rate - Binary task completion
Task progression - Furthest stage reached
Coordination quality - Bimanual synchronization
Trajectory efficiency - Path length and smoothness
Safety - Collision avoidance

See Metrics for details.

Choosing a Task#

For Learning Bimanual Coordination:

StackTwoBlocks - Basic block stacking
LiftPot - Coordinated lifting
LiftTray - Balance and coordination

For Precision and Placement:

PickSingleBookFromTable - Precise grasping and placement
StackSingleBookShelf - Shelf alignment
PackBox - Spatial reasoning

For Continuous Coordination:

RotateValve - Continuous bimanual motion
DragOverAndLiftTray - Complex multi-stage manipulation

For Hand-to-Hand Transfer:

CubeHandover - Horizontal transfer
VerticalCubeHandover - Vertical transfer

For Testing Generalization:

Use Position variations for position robustness
Use Orientation variations for rotation robustness
Use PositionAndOrientation for full randomization

Tasks Overview

Contents