Export and quantize SmolVLA for ONNX Runtime on Arm
Introduction
Set up the SmolVLA environment
Export and validate the SmolVLA ONNX model
Quantize SmolVLA to INT4 and compare it with FP32
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr
Last updated:  26 Aug 2026
Skill level:
  Advanced
Reading time:
  1 hr
Last updated:
  26 Aug 2026
Author:Tirui Wu, Arm
Arm IP:Cortex-A Neoverse
Tags: ML Linux Python PyTorch TorchAO ONNX ONNX Runtime LeRobot
Author:
  • Tirui Wu, Arm
Arm IP:
Cortex-A Neoverse
Tags:
ML Linux Python PyTorch TorchAO ONNX ONNX Runtime LeRobot

Who is this for?

This is an advanced topic for machine learning developers who want to export and quantize a vision-language-action model for ONNX Runtime on Arm.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Export SmolVLA from PyTorch as an ONNX model
  • Run and validate the FP32 ONNX model with ONNX Runtime on an Arm CPU
  • Quantize eligible linear weights to INT4 and store them in a packed ONNX model
  • Run the FP32 and INT4 models with identical inputs and compare their action outputs and ONNX Runtime latency

Prerequisites

Before starting, you will need the following:

  • An Arm Linux system, such as an Arm cloud instance or Radxa Orion O6, with Python 3.12 installed and at least 50 GB of free storage
  • Familiarity with Python, PyTorch, and Linux command-line tools

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll export SmolVLA from PyTorch to FP32 ONNX and validate it with ONNX Runtime on an Arm Linux CPU. First, you’ll set up the pinned model, source, and Python environment. Then, you’ll export the model and compare its output with PyTorch. Finally, you’ll quantize eligible linear weights to packed INT4 with TorchAO and compare FP32 and INT4 action outputs and latency using identical inputs.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
What do I need to check on my Arm machine before I run the setup scripts?
Confirm you are on an Arm Linux CPU with Python 3.12 and at least 50 GB of free storage.
Where do the scripts place the exported and quantized models?
The FP32 export is saved at work/onnx/fp32/model.onnx. The TorchAO converter writes the packed INT4 model to work/onnx/int4/smolvla-int4.onnx.
What output confirms that the ONNX export matches PyTorch?
The exporter ends with PASS: ONNX Runtime matches PyTorch within atol=0.001 and rtol=0.001. You can also inspect work/onnx/fp32/validation.json to confirm the provider, output shape, and validation result.
What exactly gets quantized, and how can I verify it?
TorchAO applies weight-only INT4 quantization to eligible constant linear weights and replaces supported MatMul and Gemm operations with packed com.microsoft::MatMulNBits operations. The converter reports how many eligible linear operations it converted.
How should I compare FP32 and INT4 runs on Arm?
Run both models with the deterministic reference batch created during export. The comparison writes an action-output figure and JSON report with the latency and overall output error.
Next