Quick Start

1. Start the Runtime Environment

This project uses Docker for containerized deployment. Start it from the project root directory:

chmod +x run.sh
sudo ./run.sh

The following environment variables can be customized:

Environment Variable	Description	Default Value
`IMAGE_NAME`	Docker image name	`ghrc_2026:v0`
`CONTAINER_NAME`	Container name	`isaac_sim_lerobot`
`HOST_WORKSPACE`	Host project directory path	Directory containing `run.sh`
`CONTAINER_WORKSPACE`	Working directory in the container	`/workspace/GlobalHumanoidRobotChallenge_2026_Baseline`
`SHM_SIZE`	Shared memory size	`8g`
`ISAAC_CACHE_ROOT`	Isaac Sim cache directory	`${HOME}/.cache/isaac_sim_container`
`HF_CACHE`	Hugging Face cache directory	`${HOME}/.cache/huggingface`
`HEADLESS`	Whether to enable headless mode	`0` (No)

Usage examples:

# Basic startup
./run.sh

# Headless mode (remote server)
./run.sh --headless

# Custom image name
IMAGE_NAME=my_custom_image:v1 ./run.sh

# Custom mount path
HOST_WORKSPACE=/my/project/path ./run.sh

⚠️ Before the first run, make sure Docker and the NVIDIA Container Toolkit have been installed.

2. Features

2.1 Teleoperation (`Teleoperate`)

Use the keyboard to teleoperate the Walker S2 robot:

# Before starting teleoperation, set teleop.evdev_device_path to the /dev/input/event* device corresponding to your keyboard

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false

# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false
  
# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_teleoperate.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --display_data=false

Parameter	Description	Default Value
`--robot.type`	Robot type	`walker_s2_sim`
`--robot.headless`	Whether to run in headless mode	`false`
`--task`	Task name (`Part_Sorting`, `Conveyor_Sorting`, `Foam_Inlaying`, or `Packing_Box`)	Required
`--teleop.type`	Teleoperation device type	`walker_s2_keyboard`
`--teleop.evdev_device_path`	Keyboard evdev device path	Auto-detected; specifying it is recommended
`--display_data`	Whether to display camera images	`false`

2.1.1 Keyboard Mapping

End-effector translation (hold the key for continuous movement)

Key	Action
`1`	Move the end effector along +Y
`3`	Move the end effector along -Y
`4`	Move the end effector along -X
`6`	Move the end effector along +X
`7`	Move the end effector along +Z (up)
`9`	Move the end effector along -Z (down)

End-effector rotation (hold the key for continuous rotation)

Key	Action
`y`	Rotate about the Y-axis in the positive direction
`u`	Rotate about the Y-axis in the negative direction
`v`	Rotate about the X-axis in the positive direction
`b`	Rotate about the X-axis in the negative direction
`n`	Rotate about the Z-axis in the positive direction
`m`	Rotate about the Z-axis in the negative direction

Gripper control

Key	Action
`k`	Open gripper
`l`	Close gripper

System control

Key	Action
`o`	Switch the controlled arm (left ↔ right)
`0`	Switch between single-arm control and synchronized dual-arm control
`h`	Return to the home position
`+` / `=`	Increase the movement speed level
`-`	Decrease the movement speed level
`q`	Exit teleoperation

2.1.2 Speed Levels

Index	Speed (m/step)	Description
0	0.010	Low speed (default)
1	0.035	Medium speed

2.2 Data Recording (`Record`)

Record human demonstration data through keyboard teleoperation:

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Part_Sorting \
    --dataset.root=datasets/Part_Sorting/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Part Sorting" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false
  
# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Conveyor_Sorting \
    --dataset.root=datasets/Conveyor_Sorting/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Conveyor Sorting" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Foam_Inlaying \
    --dataset.root=datasets/Foam_Inlaying/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Foam Inlaying" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false

# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_record.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --teleop.type=walker_s2_keyboard \
    --teleop.evdev_device_path=/dev/input/event2 \
    --dataset.repo_id=your_org/Packing_Box \
    --dataset.root=datasets/Packing_Box/record/v0 \
    --dataset.num_episodes=10 \
    --dataset.single_task="Packing Box" \
    --dataset.video=true \
    --dataset.fps=30 \
    --dataset.episode_time_s=1000000 \
    --dataset.push_to_hub=false \
    --play_sounds=false

⚠️ After running the command, Isaac Sim 5.1.0 may take some time to load. If the Isaac Sim window shows 'Not Responding', wait until loading completes. Once it has loaded successfully, you will be prompted to press Enter to begin recording. You can then use keyboard teleoperation to control both arms and manually record demonstrations one episode at a time.

⚠️ Press the left arrow key to discard the current episode. Both arms will return to their initial positions so that the same episode can be recorded again. Press the right arrow key to end the current episode early, save the data, reset the environment, and proceed to the next recording. After all recordings are complete, press Esc to stop recording entirely and save all recorded episodes.

⚠️ The collected data is saved in the directory specified by dataset.root.

Parameter	Description	Default Value / Notes
`--robot.type`	Robot type	`walker_s2_sim`
`--robot.headless`	Whether to run in headless mode	`false`
`--task`	Task name	Required
`--teleop.type`	Teleoperation device type	`walker_s2_keyboard`
`--teleop.evdev_device_path`	Keyboard evdev device path	Auto-detected; specifying it is recommended
`--dataset.repo_id`	Dataset ID in Hugging Face format	Required
`--dataset.root`	Local save path, used instead of `repo_id`	None
`--dataset.num_episodes`	Number of episodes to record	`50`
`--dataset.single_task`	Task description	Required
`--dataset.video`	Whether to record video	`true`
`--dataset.fps`	Frame rate	`30`
`--dataset.episode_time_s`	Recording duration per episode, in seconds	`60`
`--dataset.push_to_hub`	Whether to upload to Hugging Face	`false`
`--play_sounds`	Whether to play audio cues	`false`

⚠️ The play_sounds parameter must be set to false because the Docker image does not include the audio playback dependencies.

Dataset Structure

datasets/packing_box/
├── data
│   └── chunk-000
│       └── file-000.parquet
├── meta
│   ├── episodes
│   │   └── chunk-000
│   │       └── file-000.parquet
│   ├── info.json
│   ├── stats.json
│   └── tasks.parquet
└── videos
    ├── observation.images.head_left
    │   └── chunk-000
    │       └── file-000.mp4
    ├── observation.images.head_right
    │   └── chunk-000
    │       └── file-000.mp4
    ├── observation.images.wrist_left
    │   └── chunk-000
    │       └── file-000.mp4
    └── observation.images.wrist_right
        └── chunk-000
            └── file-000.mp4

2.3 Dataset Replay (`Replay`)

Replay a recorded dataset for verification:

# task1
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Part_Sorting \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Part_Sorting/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task2
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Conveyor_Sorting \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Conveyor_Sorting/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task3
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Foam_Inlaying \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Foam_Inlaying/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false
  
# task4
/isaac-sim/python.sh src/lerobot/scripts/lerobot_replay.py \
    --robot.type=walker_s2_sim \
    --robot.headless=false \
    --task=Packing_Box \
    --dataset.repo_id=your_org/your_dataset \
    --dataset.root=datasets/Packing_Box/record/v0 \
    --dataset.episode=0 \
    --play_sounds=false

Parameter	Description	Default Value / Notes
`--robot.type`	Robot type	`walker_s2_sim`
`--task`	Task name; it must match the task used for recording	Required
`--dataset.repo_id`	Dataset source	Required
`--dataset.root`	Local dataset path, used instead of `repo_id`	None
`--dataset.episode`	Index of the episode to replay, starting from 0	Required
`--play_sounds`	Whether to play audio cues	`false`

⚠️ The play_sounds parameter must be set to false because the Docker image does not include the audio playback dependencies.

Environment State Restoration Mechanism

During replay, the system automatically restores the poses of environment objects stored in the dataset to the simulation scene:

Data source: The observation.state column contains the robot state in the first 20 dimensions and the environment object poses in the remaining dimensions.
Restoration timing: After robot.connect() and before replay begins.

Extract the environment object poses from the first frame, skipping the first 20 robot-state dimensions.
Call robot.set_environment_state() to restore the object poses.
Advance the physics simulation by 50 steps to allow the objects to stabilize.

Environment state dimension calculation:

Task	Dimension Formula	Description
Part_Sorting	`num_parts × 2 × 7`	Two types of parts
Conveyor_Sorting	`num_parts × 2 × 7`	Two types of parts
Packing_Box	`num_boxes × num_parts × 7`	Multiple boxes
Foam_Inlaying	`0`	No tracked objects

Each object is represented by 7 dimensions:

[x, y, z, qx, qy, qz, qw]

1. Start the Runtime Environment​

2. Features​

2.1 Teleoperation (Teleoperate)​

2.1.1 Keyboard Mapping​

2.1.2 Speed Levels​

2.2 Data Recording (Record)​

Dataset Structure​

2.3 Dataset Replay (Replay)​

Environment State Restoration Mechanism​