.. _xarm_gripper:

Gripper-Based Robot
==========================

With a VR headset connected via ALVR and `tele.py` running, you can start controlling a robot with gripper in virtual reality.

.. code-block:: bash

    # example 
    ./example.sh peg_insertion
    # python3 tele.py --env configs/examples/maniskill_panda/peg_insertion_side.yaml --tele configs/agents/panda/tele_right.yaml


.. figure:: assets/peg_insertion_example.png
    :width: 400px
    :align: center

    PegInsertionSide-v1 environment with Panda robot. The task is to pick up a orange-white peg and insert the orange end into the box with a hole in it. See `ManiSkill3 PegInsertionSide-v1 <https://github.com/haosulab/ManiSkill/blob/83570f1e9fe7d5dee942e338d336cddd43d8718d/mani_skill/envs/tasks/tabletop/peg_insertion_side.py>`_.

How Control Works
---------------------------

.. figure:: assets/controller_map.jpg
    :width: 600px
    :align: center

    Controller Key Mapping definition

The control system maps your hand position directly to the robot's end-effector. Press the **Bottom button** on the right controller to start synchronization - the robot will follow your hand movements. Press it again to stop synchronization and freeze the robot in place.

Use the **Upper button** to control the gripper. Hold it down to close the gripper, release to open it.
.. is used to control the gripper opening and closing.

Recording Your Movements 
-------------------------------

For simulation tasks with predefined success conditions, recording happens automatically: 

- Recording **starts** when you begin synchronization with the robot.
- Recording **stops** once you complete the task successfully, plus a few extra seconds for good measure.
- The environment **resets** itself after each recording.

You also have manual controls available:

- Press the **A button** to **stop** recording manually.
- Press the **B button** to reset the environment whenever you want.

Visual Feedback
---------------------------

.. figure:: assets/peg_insertion_example_vis.png
    :width: 400px
    :align: center

    A small green shpere as visualization kit

A small green sphere shows you important information. For example, in a "pick up the cube" task, the green sphere marks exactly where you need to place the cube.

The sphere changes color to tell you what's happening:

- **Green**: Ready to start. 
- **Red**: You're synchronized with the robot (robot following your hand)
- **Blue**: Success! You've completed the task

After the sphere turns blue, the environment will automatically reset after a few seconds and the sphere returns to green.


Workflow Example
---------------------------

1. Environment starts with **green** sphere
2. Press bottom button → sphere turns **red** (synchronization active)
3. Perform your demonstration with the robot following your hand and press the upper button to close the gripper
4. Complete the task → sphere turns **blue** (success!)
5. System automatically resets after a few seconds → sphere returns to **green**

Controller Reference
---------------------------

- ``Bottom button``: Toggles synchronization between the controller and the robot's end-effector.

  - Press **once** to enable (the robot will follow your hand).
  - Press again to disable (freeze the robot in place).

- ``Upper button``: Controls the gripper.

  - Hold down to close the gripper.
  - Release to open the gripper.

- ``A button``: Manually stop recording.

- ``B button``: Manually reset the environment.

- ``Meta icon``: Adjust your VR view.

  - Hold for 3 seconds to adjust the view.