Pipeline guide · commands verbatim

The VLA workflow: collect, fine-tune, deploy

Vision-language-action (VLA) policies let a robot follow language prompts. The official pipeline has three steps — teleop data collection, Isaac-GR00T N1.7 fine-tuning, SONIC deployment. All commands below are copied from the VLA Workflow and VLA Inference tutorials.

Step 1 · Collect teleop demonstrations

Record demonstrations with VR whole-body teleoperation. The data exporter captures robot state, camera images, and teleop actions as a LeRobot dataset. Full setup (camera server, VR teleop, recording controls) is in the Data Collection tutorial.

python gear_sonic/scripts/launch_data_collection.py \
  --camera-host 192.168.123.164 \
  --task-prompt "pick up the cup"

Step 2 · Fine-tune with Isaac-GR00T

Fine-tune the GR00T N1.7 base model on your dataset using the Isaac-GR00T training pipeline. Prerequisites: clone and install Isaac-GR00T (uv sync --all-extras), a multi-GPU machine (4+ GPUs recommended), and the dataset accessible from the training machine.

git clone https://github.com/NVIDIA/Isaac-GR00T.git
cd Isaac-GR00T
uv sync --all-extras

Launch fine-tuning:

export NUM_GPUS=4
uv run python \
    gr00t/experiment/launch_finetune.py \
    --base-model-path nvidia/GR00T-N1.7-3B \
    --dataset-path /path/to/your/collected_dataset \
    --embodiment-tag UNITREE_G1_SONIC \
    --modality-config-path gr00t/configs/data/embodiment_configs.py \
    --num-gpus $NUM_GPUS \
    --output-dir /path/to/output \
    --save-total-limit 5 \
    --save-steps 5000 \
    --max-steps 20000 \
    --use-wandb \
    --global-batch-size 32 \
    --color-jitter-params brightness 0.3 contrast 0.4 saturation 0.5 hue 0.08 \
    --dataloader-num-workers 4

Key parameters: --max-steps (20k is a good starting point), --global-batch-size (total across all GPUs), --save-steps (checkpoint interval), --use-wandb (training metrics to Weights & Biases). Checkpoints land in --output-dir (checkpoint-5000 … checkpoint-20000 + config.json + processor_config.json).

Step 3 · Deploy for inference

Start the PolicyServer on the GPU machine (from the Isaac-GR00T repo):

uv run python gr00t/eval/run_gr00t_server.py \
    --model-path /path/to/output/checkpoint-20000 \
    --embodiment-tag UNITREE_G1_SONIC \
    --device cuda:0 \
    --port 5550

Then run inference from the GR00T-WholeBodyControl repo:

python gear_sonic/scripts/launch_inference.py \
    --policy-host <gpu_machine_ip> \
    --policy-port 5550 \
    --camera-host 192.168.123.164 \
    --prompt "pick up the soda can and place it in the bin"

The inference environment is installed with bash install_scripts/install_inference.sh (creates .venv_inference), the camera server runs as a systemd service on the robot, and gear_sonic_deploy must be built. For the low-latency checkpoint, download deployment files with python download_from_hf.py --low-latency and launch gear_sonic_deploy with the low-latency model prefix and matching observation config. For SONIC v1.1 (robot-heading-normalized), use python download_from_hf.py --sonic-v1-1.

VLA workflow FAQ

What is a VLA workflow in the SONIC stack?
It is the full pipeline to make a robot act on language commands: 1) collect teleop demonstrations with VR whole-body teleoperation, 2) fine-tune the Isaac-GR00T N1.7 foundation model on that data, 3) deploy the finetuned policy for autonomous inference with the SONIC controller.
What does the VLA inference stack look like?
Four cooperating pieces: the Isaac-GR00T PolicyServer (serves actions over ZMQ, runs on a GPU machine), the VLA inference client (reads camera + robot state, queries the server, publishes actions), the C++ deploy binary (executes whole-body control), and the camera server (publishes images over ZMQ, runs on the robot as a systemd service).
Which SONIC checkpoint should I use for VLA?
Use the sonic_v1_1 checkpoint when your VLA policy was trained against the robot-heading-normalized SONIC controller (it uses a 10-frame SMPL/wrist reference horizon and wrist-pose augmentation, and is not the low-latency checkpoint). For the most responsive teleoperation, the low-latency checkpoint pairs with matching observation config.