Guide · from the official tutorials

Teleoperating the G1: VR, keyboard, gamepad, camera

Whole-body teleoperation on the SONIC stack runs through the C++ deploy binary's input types, plus the VR pipeline that streams SMPL poses. Pick the input that fits your task — this page maps the options from the official teleoperation guide and tutorials.

The input types

Input typeWhat it is
keyboardDirect keyboard input (default)
gamepadWireless controller
gamepad_managerGamepad + quick switching to ZMQ/ROS2
zmqNetwork motion streaming
zmq_managerDynamic switching between planner and network motion streaming
managerDynamic switching between keyboard, gamepad, ZMQ, and ROS2
ros2ROS2 topic control (requires planner, only if built with ROS2 support)

Pass any of these with --input-type <type> to the deploy binary. The full argument reference (ZMQ host/port/topic, output type, compliance, logging flags) is in the C++ deployment program-flow doc.

VR whole-body teleoperation (PICO)

The flagship method: a PICO headset drives full-body motion via the pico_manager_thread_server.py streamer, which publishes SMPL body parameters over ZMQ (port 5556, topic pose). The C++ deploy subscribes with zmq_manager. Setup details (PICO calibration, .venv_teleop) are in the VR teleop setup and data collection docs. The official safety note: keep a clear 3-meter safety zone around the robot and a safety operator ready to press O for emergency stop.

C++ deploy commands

cd gear_sonic_deploy
source scripts/setup_env.sh
./deploy.sh --input-type zmq_manager real

real targets the physical robot; sim targets MuJoCo (--disable-crc-check is required for simulation). The low-latency checkpoint uses the low-latency model prefix and matching observation config (see SONIC model selector).

ZMQ configuration on the deploy binary: --zmq-host (default localhost), --zmq-port (default 5556), --zmq-topic (default pose), --zmq-out-port (default 5557), --zmq-out-topic (default g1_debug). CloudXR / DeviceIO bring-up is documented in the Isaac Teleop Setup tutorial in the repo.

Teleoperation FAQ

What teleoperation methods does the SONIC stack support?
The C++ deploy binary accepts several input types: keyboard, gamepad (wireless controller), gamepad_manager (with quick switching to ZMQ/ROS2), zmq (network motion streaming), zmq_manager (dynamic switching between planner and streamed motion), manager (dynamic switching between keyboard, gamepad, ZMQ, and ROS2), and ros2 (ROS2 topic control, requires planner). VR whole-body teleoperation uses the PICO headset with the pico_manager streamer feeding SMPL poses over ZMQ.
What is ZMQ's role in teleoperation?
ZMQ carries the live motion stream. With --input-type zmq the deploy binary subscribes to a pose topic (default port 5556, topic 'pose'), and with zmq_manager it can switch between planner-generated motion and network-streamed motion. Control results publish back on port 5557 (topic g1_debug) with --output-type zmq.
How do I run C++ deployment for teleop?
From gear_sonic_deploy/: source scripts/setup_env.sh, then ./deploy.sh --input-type zmq_manager real for real robot (or sim for MuJoCo). For the low-latency checkpoint use the matching model prefix and observation config.