ImVR: Immersive VR Teleoperation System for General Purpose

Yulin Liu1, Zihao He2, Fanbo Xiang3, Runlin Guo3, Zhiao Huang3, Jialin Zhang3, Bo Ai1 Stone Tao1 Hao Su3
1University California, San Diego 2Shanghai Jiao Tong University 3Hillbot Inc.

Our system places the operator in a fully 3D virtual world for an intuitive and immersive teleoperation experience. It is designed for general-purpose use, supporting a variety of robot setups, including single- and dual- arm configurations, grippers, three-fingered hands, dexterous hands, and both simulation and real-world environments.

Abstract

We present ImVR, an immersive VR-based teleoperation system that places the operator in a fully 3D virtual environment for intuitive and precise robot control. Designed for general-purpose use, ImVR supports a wide range of robot configurations—including single- and dual-arm setups, parallel grippers and multi-fingered dexterous hands—while operating seamlessly in both simulation and the real world. It ensures absolute alignment between the operator’s hand and the robot’s end-effector, offers adaptive viewpoints to enhance situational awareness and precision, and enables easy deployment across diverse robotic platforms. Experiments and user studies highlight ImVR’s versatility, ease of use, and effectiveness in accelerating data collection, making it a powerful tool for large-scale robot learning and real-world manipulation tasks.

Placing Human in 3D Virtual World

Left: The first-person view of the operator in the 3D virtual world.
Right: The third-person view of trajectory recorded.
Our system enables adaptive viewpoints to focus on precise manipulation tasks while maintaining smooth, low-latency experience.

BibTeX

@article{todo
      
}