Quick Start

1. Start the Runtime Environment

This project is deployed with Docker containers. Start it from the project root:

chmod +x run.sh
sudo ./run.sh

Custom environment variables are supported:

Environment Variable	Description	Default
`IMAGE_NAME`	Docker image name	`ghrc_2026:v0`
`CONTAINER_NAME`	Container name	`isaac_sim_lerobot`
`HOST_WORKSPACE`	Project directory path on the host	Directory where `run.sh` is located
`CONTAINER_WORKSPACE`	Working directory path inside the container	`/workspace/GlobalHumanoidRobotChallenge_2026_Baseline`
`SHM_SIZE`	Shared memory size	`8g`
`ISAAC_CACHE_ROOT`	Isaac Sim cache directory	`${HOME}/.cache/isaac_sim_container`
`HF_CACHE`	Hugging Face cache directory	`${HOME}/.cache/huggingface`
`HEADLESS`	Whether to enable headless mode	`0` (No)

Examples:

# Basic start
./run.sh

# Headless mode (remote server)
./run.sh --headless

# Custom image name
IMAGE_NAME=my_custom_image:v1 ./run.sh

# Custom mount path
HOST_WORKSPACE=/my/project/path ./run.sh

⚠️ Before the first run, make sure you have installed Docker and NVIDIA Container Toolkit.

2. Modules

2.1 Teleoperate

Use the keyboard to teleoperate the Walker S2 robot:

# Before starting Teleoperate, you must set teleop.evdev_device_path to the correct event device for your keyboard.

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false

# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false
  
# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false

Argument	Description	Default
`--robot.type`	Robot type	`walker_s2_sim`
`--robot.headless`	Whether to run in headless mode	`false`
`--task`	Task name (Part_Sorting, Conveyor_Sorting, Foam_Inlaying, Packing_Box)	`Foam_Inlaying`
`--teleop.type`	Teleoperation device type	`walker_s2_keyboard`
`--teleop.evdev_device_path`	Keyboard evdev device path	Auto-scan (recommended to set explicitly)
`--display_data`	Whether to display camera frames	`false`

2.1.1 Key Mapping

End-effector translation (hold to move continuously)

Key	Action
`1`	Move end-effector in +Y
`3`	Move end-effector in -Y
`4`	Move end-effector in -X
`6`	Move end-effector in +X
`7`	Move end-effector in +Z (up)
`9`	Move end-effector in -Z (down)

End-effector rotation (hold to rotate continuously)

Key	Action
`y`	Rotate around +Y
`u`	Rotate around -Y
`v`	Rotate around +X
`b`	Rotate around -X
`n`	Rotate around +Z
`m`	Rotate around -Z

Gripper control

Key	Action
`k`	Open gripper
`l`	Close gripper

System control

Key	Action
`o`	Switch control arm (left ↔ right)
`0`	Switch single-arm / dual-arm synchronous control mode
`h`	Move to home position
`+` / `=`	Increase speed level
`-`	Decrease speed level
`q`	Quit teleoperation

2.1.2 Speed Levels

Index	Speed (m/step)	Notes
0	0.010	Low speed (default)
1	0.035	Medium speed

2.2 Record

Record human demonstration data through keyboard teleoperation:

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Part_Sorting \
    --dataset.root=datasets/Part_Sorting/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Part Sorting" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false
  
# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Conveyor_Sorting \
    --dataset.root=datasets/Conveyor_Sorting/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Conveyor Sorting" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Foam_Inlaying \
    --dataset.root=datasets/Foam_Inlaying/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Foam Inlaying" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false

# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Packing_Box \
    --dataset.root=datasets/Packing_Box/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Packing Box" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false

⚠️ After running the command, Isaac Sim 5.1.0 will take some time to load. If a dialog shows Isaac Sim is not responding, please wait until it finishes loading. Once loaded, you will be prompted to press Enter to start recording. You can then teleoperate the dual arms with the keyboard and record episodes one by one manually.

⚠️ Press the left arrow key to discard the current episode. The arms will return to the initial pose and you can re-record the same one. Press the right arrow key to end the current episode early, save the data, and enter the environment reset stage to proceed to the next recording. After finishing, press Esc to stop recording completely and save all recorded episodes.

⚠️ The collected data is saved under the dataset.root directory.

Argument	Description	Default / Notes
`--robot.type`	Robot type	`walker_s2_sim`
`--robot.headless`	Whether to run in headless mode	`false`
`--task`	Task name	Required
`--teleop.type`	Teleoperation device type	`walker_s2_keyboard`
`--teleop.evdev_device_path`	Keyboard evdev device path	Auto-scan (recommended to set explicitly)
`--dataset.repo_id`	Dataset ID (Hugging Face format)	Required
`--dataset.root`	Local save path (alternative to repo_id)	`None`
`--dataset.num_episodes`	Number of episodes to record	`50`
`--dataset.single_task`	Task description	Required
`--dataset.video`	Whether to record video	`true`
`--dataset.fps`	FPS	`30`
`--dataset.episode_time_s`	Recording time per episode (seconds)	`60`
`--dataset.push_to_hub`	Whether to upload to Hugging Face	`false`
`--play_sounds`	Whether to play prompt sounds	`false`

⚠️ play_sounds must be set to false because the image does not include audio playback dependencies.

Dataset Structure

datasets/packing_box/
├── data
│   └── chunk-000
│       └── file-000.parquet
├── meta
│   ├── episodes
│   │   └── chunk-000
│   │       └── file-000.parquet
│   ├── info.json
│   ├── stats.json
│   └── tasks.parquet
└── videos
    ├── observation.images.head_left
    │   └── chunk-000
    │       └── file-000.mp4
    ├── observation.images.head_right
    │   └── chunk-000
    │       └── file-000.mp4
    ├── observation.images.wrist_left
    │   └── chunk-000
    │       └── file-000.mp4
    └── observation.images.wrist_right
        └── chunk-000
            └── file-000.mp4

2.3 Replay

Replay a recorded dataset for verification:

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Part_Sorting/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Conveyor_Sorting/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Foam_Inlaying/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Packing_Box/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false

Argument	Description	Default / Notes
`--robot.type`	Robot type	`walker_s2_sim`
`--task`	Task name (must match the recording)	Required
`--dataset.repo_id`	Dataset source	Required
`--dataset.root`	Local dataset path (alternative to repo_id)	None
`--dataset.episode`	Episode index to replay (starting from 0)	Required
`--play_sounds`	Whether to play prompt sounds	`false`

⚠️ play_sounds must be set to false because the image does not include audio playback dependencies.

Environment State Restore Mechanism

During replay, the system automatically restores the poses of environment objects into the simulation scene:

Data source: the observation.state column contains the robot state (first 20 dimensions) and the environment object poses (subsequent dimensions).
When: after robot.connect() and before starting replay.
Procedure:
- Extract environment object poses from the first frame (skip the first 20 dimensions of robot state).
- Call robot.set_environment_state() to restore object poses.
- Step the physics simulation for 50 steps to let objects settle.

Environment state dimension calculation:

Task	Dimension formula	Notes
Part_Sorting	`num_parts × 2 × 7`	Two types of parts
Conveyor_Sorting	`num_parts × 2 × 7`	Two types of parts
Packing_Box	`num_boxes × num_parts × 7`	Multiple boxes
Foam_Inlaying	`0`	No tracked objects

Each object is 7D: [x, y, z, qx, qy, qz, qw]

1. Start the Runtime Environment​

2. Modules​

2.1 Teleoperate​

2.1.1 Key Mapping​

2.1.2 Speed Levels​

2.2 Record​

Dataset Structure​

2.3 Replay​

Environment State Restore Mechanism​