Customize Env#

python3 tele.py --env <env_config> --tele <tele_config>

The teleop script takes in two configurations: one for environment and one for teleoperation.

The teleoperation config is tightly coupled with robot configurations – each robot has its own teleoperation config.

To use the same robot across different environments, you simply need to create a new environment in ManiSkill3 and modify the environment config accordingly.

Why this design?

It decouples robot control logic from the task/environment logic, so that in real-world teleoperation, the same robot and control interface can be reused consistently, while only the environment-specific settings need to be changed.

Environment Config#

# these definitions follow ManiSkill3 definition.
env_name: str = "PickCube-v1" # name of the environment
robot_uids: str | None = None # name of the robot

ignore_terminations: bool = True
# ignore the terminations for all mani_skill envs, you can skip it.

obs_mode: str = "state"
action_mode: str | None = None
render_mode: str = "rgb_array"
sim_backend: str = "cpu"

control_mode: str = "pd_pos_control"
sim_config: NewSimConfig = field(default_factory=NewSimConfig)

# these are teleoperation related definations.
root_pose: PoseConfig = PoseConfig()
# Root pose of VR display - changes your VR view
goal_site: PoseConfig = PoseConfig(p=[0.0, 0.0, 1.0])
# Pose of green sphere for recording status visualization
# If the environment has a goal site defined in ManiSkill3 Env (e.g. goal position for pick up task),
# it will use that goal site pose instead of this one

success_condition: bool = True
# whether have success condition or not
auto_record_success: bool = False
# If True, recording stops automatically when success condition is met
auto_record_success_threshold: int = 20
# Control steps to wait before stopping recording after success
# For example, at 10 Hz control frequency, recording stops 2 seconds after success

You can preview the environment by provided tools:

python3 tools view <env_config>

See example configs in vr_teleop/configs/examples.