Perform Quantization

The following command downloads 100 images from Open Images v7 (approximately 34MB). You will use these to calibrate during the quantization.

Run the command on your development machine:

    

        
        
module-2-cnn-inference-with-quantization/scripts/aot/create_calibration_images.py \
  --data-dir datasets \
  --out module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/calibration_images \
  --count 100

    

You should see output similar to this:

    

        
        
Downloading 100 Open Images validation images to datasets/open-images-v7-validation...
Verified 100 Open Images files.
Wrote 100 calibration images across 20 categories to module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/calibration_images

    

The script downloads five images from each of 20 broad object categories and verifies every file with SHA256. A production calibration set should represent the cameras, lighting, composition, and objects that the deployed application will receive; this set is purely to demonstrate the workflow.

Quantize and lower the model

    

        
        
module-2-cnn-inference-with-quantization/scripts/aot/quantize_mobilenetv2_xnnpack.py \
  --spec module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/model_spec.json \
  --calibration-images module-2-cnn-inference-with-quantization/artifacts/mobilenetv2/calibration_images \
  --num-calib 100

    

The script runs PT2E post-training quantization with the XNNPACK quantizer, and calibrates the prepared graph, converts it, lowers it to ExecuTorch with XNNPACK, and writes the INT8 .pte artifact.

You should receive a quantized artifact of similar size to that shown in the table below:

In this run, the INT8 artifact was 7.59 MB rather than 13.99 MB, a reduction of approximately 46%.

An INT8 value uses one quarter of the storage of an FP32 value. That ratio does not apply automatically to the complete .pte artifact. The file also contains graph and backend metadata, not every component is quantized, and this lab leaves the classifier head in FP32 by default.

Understand the code

The script configures the quantizer as follows:

    

        
        
quantizer = XNNPACKQuantizer()
quantizer.set_global(get_symmetric_quantization_config(is_per_channel=True))

if not args.quantize_classifier_head:
    quantizer.set_filter_function(skip_classifier_head)

    

The global configuration requests symmetric per-channel weight quantization where supported, using is_per_channel=True. Per-channel quantization gives each output channel its own scale, helping preserve precision when convolution filters have different weight ranges. Activations use one scale per tensor, determined during calibration.

By default, the script leaves the final classifier head in FP32. The classifier head is the last layer that maps MobileNetV2 features to ImageNet class scores. This is a conservative compromise between optimization and classification stability: most of the model’s compute is in the convolutional feature extractor, while the classifier head directly produces the final ImageNet scores. You can optionally pass --quantize-classifier-head as a follow-up experiment. It may reduce size or latency a little more, but rerun the delegation, latency, and top-5 checks because the final scores may shift.

Next, the script prepares the exported model for calibration by adding observer modules:

    

        
        
exported = export(model, example_inputs)
prepared = prepare_pt2e(exported.module(), quantizer)
move_exported_model_to_eval(prepared)

    

The observers collect values during calibration; their presence alone does not make inference faster. The calibration loop supplies the observed ranges:

    

        
        
for index, (image, _) in enumerate(calibration_loader):
    if index >= args.num_calib:
        break
    prepared(image)

    

Calibration quality depends on input quality. A set with different lighting, composition, preprocessing, or subject matter from the deployed workload can produce activation ranges that do not represent real use. Module 1 did not include this step because its transformer comparisons remained in FP32.

After calibration, conversion creates the quantized model. The script exports and lowers that converted graph:

    

        
        
quantized_model = convert_pt2e(prepared)
exported_quantized = export(quantized_model, example_inputs)
edge = to_edge_transform_and_lower(
    exported_quantized,
    partitioner=[XnnpackPartitioner()],
    compile_config=EdgeCompileConfig(_check_ir_validity=False),
)
program = edge.to_executorch()

    

The resulting graph is no longer the original FP32 program. It contains the quantized operations and delegated regions that ExecuTorch will execute.

Optional further reading