Create the INT8 Ethos-U artifact

Download 100 images from Open Images v7 (approximately 34MB). You will use these to calibrate during the quantization. If you completed Module 2 or 3 on this machine, the command verifies and reuses the files in datasets/open-images-v7-validation.

Run the script on your Ubuntu machine:

    

        
        
python module-4-ethos-npu-accelerated-inference/scripts/aot/create_calibration_images.py \
  --data-dir datasets \
  --out module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/calibration_images \
  --count 100

    

The script downloads five images from each of 20 broad object categories and verifies every file with SHA256. A production calibration set should represent the cameras, lighting, composition, and objects expected by the deployed application. This set is purely to demonstrate the workflow but is not representative enough for production calibration.

Quantize and lower the model

The quantization setup uses EthosUQuantizer. This prepares the model for post-training quantization (PTQ).

    

        
        
compile_spec = EthosUCompileSpec(
    target=args.target,
    system_config=args.system_config,
    memory_mode=args.memory_mode,
    extra_flags=args.extra_flag,
)

quantizer = EthosUQuantizer(compile_spec)
quantizer.set_global(get_symmetric_quantization_config())
prepared = prepare_pt2e(graph_module, quantizer)

    

This module uses static W8A8 post-training quantization for supported operations. Calibration determines activation scales before compilation, and those scales remain fixed during inference. Quantized computation is required for Ethos-U acceleration. Unlike XNNPACK, the current ExecuTorch Ethos-U backend does not support dynamic activation quantization.

EthosUQuantizer applies quantization rules for the selected Ethos-U compile specification. An INT8 model prepared for XNNPACK is not automatically suitable for Ethos-U, because the backends accept different operator patterns and quantization configurations. See the Ethos-U quantization documentation for supported schemes.

The calibration loop is the same PTQ pattern introduced in Module 2:

    

        
        
for index, (image, _) in enumerate(calibration_loader):
    if index >= args.num_calib:
        break
    prepared(image)

    

After calibration, the script converts the model, exports the quantized graph, and lowers through the Ethos-U partitioner:

    

        
        
quantized_model = convert_pt2e(prepared)
exported_quantized = export_no_guards(quantized_model, example_inputs)
edge = lower_to_ethosu(exported_quantized, args)

    

Some quantize and dequantize nodes can remain around the Ethos-U delegate boundary. Before packaging, the script replaces those boundary q/dq nodes with Cortex-M-compatible equivalents so to_executorch() can write the final .pte artifact:

    

        
        
edge = replace_boundary_quant_nodes(edge)

    

The script packages the lowered edge program as an ExecuTorch program:

    

        
        
program = edge.to_executorch(
    config=ExecutorchBackendConfig(extract_delegate_segments=False)
)

    

Run on your Ubuntu machine:

    

        
        
python module-4-ethos-npu-accelerated-inference/scripts/aot/quantize_lower_ethosu.py \
  --spec module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/model_spec.json \
  --calibration-images module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/calibration_images \
  --num-calib 100 \
  --target ethos-u85-256 \
  --system-config Ethos_U85_SYS_DRAM_Mid \
  --memory-mode Shared_Sram \
  --validate-image module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/sample.jpg

    

The --validate-image option runs the FP32 model and converted PT2E INT8 graph directly with PyTorch on the Linux host, before Ethos-U lowering. It does not run the delegated .pte artifact; you will do that later on the FVP. Both models should place Labrador retriever and golden retriever near the top. This comparison checks that quantization has not caused an obvious loss of prediction quality on the sample image, and can also expose incorrect weights, preprocessing, or calibration before backend compilation.

The prediction comparison should look similar to this:

Both models classify the sample as a Labrador retriever. The higher INT8 confidence on this image does not show that INT8 improves accuracy across a dataset; it only shows how quantization changed this output distribution due to the change in numerical precision.

The generated INT8 .pte artifact should be about 3.59 MB, and the terminal should print an ExecuTorch delegation summary similar to:

    

        
        
PTE size: 3.59 MB

Delegation
==========
Ethos-U delegate call sites: 1
Fallback call_function ops:  5

    

The small fallback count includes boundary q/dq, tensor-access, and allocation operations around the delegate. The important result is the single large Ethos-U delegate call containing the main MobileNetV2 workload.

Vela, the Ethos-U compiler, prints the NPU assignment before ExecuTorch writes the .pte artifact. For MobileNetV2, you should expect a summary like this:

    

        
        
Total SRAM used                               1474.72 KiB
Total DRAM used                               3494.75 KiB
CPU operators = 0 (0.0%)
NPU operators = 56 (100.0%)
Total SRAM bandwidth            per input     30.84 MB/inference (batch size 1)
Total DRAM bandwidth            per input      3.44 MB/inference (batch size 1)
Neural network macs                         300988800 MACs/batch

    

Intrepret the Vela report

Vela prints one report for each graph region sent to it. For INT8 MobileNetV2, there is one delegated subgraph, hence one report:

• CPU and NPU operators: these counts describe the region Vela compiled. NPU operators = 56 and CPU operators = 0 mean all operators in that region were assigned to the NPU. The CPU still runs ExecuTorch and the application around the delegate.
• Neural network MACs: the MAC count estimates the amount of neural compute for one input. It is not an inference-time measurement.
• Memory area usage: the report shows how weights, constants, and feature maps are placed in the memory areas defined by the system configuration and memory mode.
• Bandwidth estimates: Vela estimates reads and writes between SRAM and off-chip memory. High traffic can indicate a memory-bound configuration, but the values are compiler estimates rather than measurements from a board.

You can use the report to check operator support and memory planning for the selected target, but final performance and energy measurements require the intended hardware.