Fine-tune SmolVLA for an SO-101 pick-and-place task on an NVIDIA DGX Spark
Introduction
Understand the SmolVLA fine-tuning workflow
Install LeRobot and prepare a Python environment
Connect the SO-101 and cameras
Calibrate and teleoperate the SO-101
Record and validate a pick-and-place dataset
Fine-tune SmolVLA with the recorded SO-101 demonstrations
Evaluate the fine-tuned SmolVLA model
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  32 hrs
Last updated:  24 Aug 2026
Skill level:
  Advanced
Reading time:
  32 hrs
Last updated:
  24 Aug 2026
Author:Koki Mitsunami, Arm
Arm IP:Cortex-X Cortex-A
Tags: ML Linux Python PyTorch LeRobot SmolVLA Hugging Face
Author:
  • Koki Mitsunami, Arm
Arm IP:
Cortex-X Cortex-A
Tags:
ML Linux Python PyTorch LeRobot SmolVLA Hugging Face

Who is this for?

This is an advanced topic for robotics and AI developers who want to train a vision-language-action model from their own SO-101 demonstrations.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Set up a LeRobot environment for SO-101 data collection and SmolVLA training.
  • Connect, calibrate, and teleoperate an SO-101 leader-follower pair with cameras.
  • Record and inspect a pick-and-place dataset, then optionally upload it.
  • Fine-tune and physically evaluate a SmolVLA model on an NVIDIA DGX Spark.

Prerequisites

Before starting, you will need the following:

  • An NVIDIA DGX Spark with at least 30 GB of free storage
  • An assembled SO-101 leader and follower, two USB cameras, and an unobstructed workspace
  • A vial or similar graspable object and a stable rack for the placement target
  • A black task mat or similarly high-contrast pickup surface
  • A Hugging Face account if you want to upload the dataset

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll collect SO-101 pick-and-place demonstrations, fine-tune SmolVLA with LeRobot, and evaluate the resulting policy on an Arm-based NVIDIA DGX Spark. First, you’ll set up Python, identify the USB devices, calibrate the leader-follower pair, and verify teleoperation. Next, you’ll record and review multi-camera episodes and fine-tune the model, then assess its control of the physical pick-and-place task.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I know the robot and cameras are mapped to the right devices before calibration?
Set ROBOT_PORT, LEADER_PORT, GRIPPER_CAMERA_ID, and WORKSPACE_CAMERA_ID in the terminal that you’ll use for calibration. If you reconnect a USB device or open a new terminal, repeat device discovery and export the current paths before proceeding.
Which components do I connect where when wiring the setup?
Connect the SO-101 leader, SO-101 follower, gripper camera, and workspace camera to the DGX Spark over USB. Mount the gripper camera on the follower, and keep the workspace camera fixed to frame the follower, pickup area, rack, and the arm’s full range of motion.
What should I check during calibration so teleoperation mirrors the leader correctly?
Start each arm with its joints near the middle of their usable ranges. Move each requested joint slowly through its safe range, support the arm, and stop before reaching limits.
When is the workspace ready to start recording demonstrations?
After calibrating and verifying teleoperation, arrange the two-camera workspace. Lay the vial on the black mat, place the rack beside it within the follower’s calibrated reach, and leave enough space between them for grasping and movement.
What result should I expect after fine-tuning SmolVLA with my demonstrations?
You’ll get a SmolVLA model adapted to your demonstrations. Evaluate it with LeRobot on the Arm-based NVIDIA DGX Spark, then use it to control the robot on the vial-to-rack pick-and-place task under the recorded conditions.
Next