Deploy INT8 SqueezeNet with ONNX Runtime on Azure Cobalt 100
Introduction
Overview
Create an Arm-based Azure Cobalt 100 virtual machine
ONNX Installation
Baseline Testing
Benchmark ONNX runtime performance with onnxruntime_perf_test
Next Steps
Deploy INT8 SqueezeNet with ONNX Runtime on Azure Cobalt 100
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Select
D4ps_v6 from the Dpsv6 series powered by Azure Cobalt 100. Use the Ubuntu Pro 24.04 LTS image and select Arm64 as the VM architecture.Activate your Python virtual environment, have the
baseline.py script ready, and ensure that the SqueezeNet INT8 model file is available as squeezenet-int8.onnx. Place the model in the same working directory as the script or provide its full path.The script completes without errors and reports a timing result for a single inference through the SqueezeNet INT8 model. Successful execution confirms that ONNX Runtime is functioning on the VM.
Use
onnxruntime_perf_test after validating the Python baseline to capture more detailed performance statistics. Expect a summary of inference metrics that you can use to evaluate ONNX Runtime efficiency on the Azure Arm64 instance.No. The script uses data in
float32 with shape (1, 3, 224, 224). This represents one input with three color channels and dimensions of 224 by 224 pixels. You’ll use this synthetic input to measure inference latency.