Estimate neural accelerator performance with ML Inference Advisor
Introduction
Understand MLIA and the NX Performance Estimator
Set up your MLIA environment
Download a public baseline model
Check baseline operator compatibility
Estimate baseline performance and inspect reports
Compare a smaller model under the same target profile
(Optional) Run and profile the model on an NX device
Next Steps
Estimate neural accelerator performance with ML Inference Advisor
Introduction
Understand MLIA and the NX Performance Estimator
Set up your MLIA environment
Download a public baseline model
Check baseline operator compatibility
Estimate baseline performance and inspect reports
Compare a smaller model under the same target profile
(Optional) Run and profile the model on an NX device
Next Steps
Choose a pretrained baseline
Download an existing quantized export rather than retraining or calibrating a model. Use mobilenet_v1_0.25_224_quant.tflite from the
TensorFlow MobileNet V1 model catalog
. It’s a public pretrained image classifier with a width multiplier of 0.25 and a fixed input shape of [1, 224, 224, 3].
The model has quantized eight-bit weights and activations, with unsigned eight-bit input and output tensors. It isn’t a neural upscaling model. However, it provides a real convolutional workload for learning the Arm ML Inference Advisor (MLIA) analysis workflow.
Download and verify the model
From mlia-demo, download the versioned public archive, and extract the LiteRT model. Name your local copy baseline.tflite:
mkdir -p models runs
curl -fL https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_0.25_224_quant.tgz -o models/baseline.tgz
tar -xzf models/baseline.tgz -C models ./mobilenet_v1_0.25_224_quant.tflite
mv models/mobilenet_v1_0.25_224_quant.tflite models/baseline.tflite
echo "ce8f61fe3e4e29560c4ea8038e4b91eb354be330fc428348b9645bd1e7de03bf models/baseline.tflite" | sha256sum --check
The expected output is:
models/baseline.tflite: OK
MLIA runs the installed converter and saves the intermediate artifacts with its reports. Keep the downloaded model in your working directory.
What you’ve accomplished and what’s next
You’ve downloaded and verified the quantized MobileNet baseline.
Next, you’ll check whether its operators are supported by passing models/baseline.tflite directly to MLIA.