Export and quantize SmolVLA for ONNX Runtime on Arm
Introduction
Set up the SmolVLA environment
Export and validate the SmolVLA ONNX model
Quantize SmolVLA to INT4 and compare it with FP32
Next Steps
Export and quantize SmolVLA for ONNX Runtime on Arm
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Confirm you are on an Arm Linux CPU with Python 3.12 and at least 50 GB of free storage.
The FP32 export is saved at
work/onnx/fp32/model.onnx. The TorchAO converter writes the packed INT4 model to work/onnx/int4/smolvla-int4.onnx.The exporter ends with
PASS: ONNX Runtime matches PyTorch within atol=0.001 and rtol=0.001. You can also inspect work/onnx/fp32/validation.json to confirm the provider, output shape, and validation result.TorchAO applies weight-only INT4 quantization to eligible constant linear weights and replaces supported
MatMul and Gemm operations with packed com.microsoft::MatMulNBits operations. The converter reports how many eligible linear operations it converted.Run both models with the deterministic reference batch created during export. The comparison writes an action-output figure and JSON report with the latency and overall output error.