Give feedback

How would you rate this Learning Path?

Review frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
What do I need to check on my Arm machine before I run the setup scripts?
Confirm you are on an Arm Linux CPU with Python 3.12 and at least 50 GB of free storage.
Where do the scripts place the exported and quantized models?
The FP32 export is saved at work/onnx/fp32/model.onnx. The TorchAO converter writes the packed INT4 model to work/onnx/int4/smolvla-int4.onnx.
What output confirms that the ONNX export matches PyTorch?
The exporter ends with PASS: ONNX Runtime matches PyTorch within atol=0.001 and rtol=0.001. You can also inspect work/onnx/fp32/validation.json to confirm the provider, output shape, and validation result.
What exactly gets quantized, and how can I verify it?
TorchAO applies weight-only INT4 quantization to eligible constant linear weights and replaces supported MatMul and Gemm operations with packed com.microsoft::MatMulNBits operations. The converter reports how many eligible linear operations it converted.
How should I compare FP32 and INT4 runs on Arm?
Run both models with the deterministic reference batch created during export. The comparison writes an action-output figure and JSON report with the latency and overall output error.

Share

Tell others about this Learning Path.