Why MobileNetV2 is appropriate for edge CPU inference

MobileNetV2 uses depthwise separable and pointwise convolutions, which commonly map to optimized CPU libraries. XNNPACK provides optimized kernels for supported operations, so a convolution-heavy model can often be lowered into a small number of large delegated regions. Compared with the transformer graphs in Module 1, this can mean:

• fewer backend boundaries
• less work on the fallback path
• larger portions of the model handled by XNNPACK
• a greater opportunity to reduce latency

INT8 quantization changes model weights and, where supported, activations from FP32 to a lower-precision representation. This reduces storage and memory-bandwidth requirements and can make optimized integer kernels available.

Quantization introduces numerical approximation, so confidence scores and occasionally prediction rankings can change. After quantization, it is important to assess whether the model still gives correct results for the particular chosen use-case, for example, Image Classification.