Benchmark the INT8 artifact on Raspberry Pi

On your development machine, copy the new artifact to the Raspberry Pi. Substitute the username and IP address in the following command:

    

        
        
scp module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/mobilenetv2_int8_xnnpack.pte \
  <user>@<pi>:~/executorch_on_arm/module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/

    

On the Raspberry Pi, and from the repo root, run the benchmark:

    

        
        
module-2-cnn-inference-with-quantization/scripts/pi/benchmark_mobilenetv2.py \
  --runtime executorch \
  --backend int8_xnnpack \
  --spec ~/executorch_on_arm/module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/model_spec.json \
  --pte ~/executorch_on_arm/module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/mobilenetv2_int8_xnnpack.pte \
  --image ~/executorch_on_arm/module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/sample.jpg \
  --runs 50 \
  --warmups 10 \
  --out ~/executorch_on_arm/module-2-cnn-inference-with-quantization/results/mobilenetv2_int8_xnnpack.json

    

Quantization changes confidence values and may change the lower-ranked classes, but you should expect that Labrador retriever and golden retriever remain the leading predictions.

Your latency result will be similar to the following:

Mean latency fell from 10.89 ms for FP32 ExecuTorch with XNNPACK to 4.01 ms for the INT8 path. Relative to the 72.55 ms PyTorch eager reference, the INT8 path was approximately 18.1 times faster in this run.