How operator support changes delegation

Local Response Normalization (LRN) rescales each activation using values from nearby channels at the same spatial position. It was used in older CNNs to reduce the relative strength of activations surrounded by other strong responses, and it is not part of standard MobileNetV2. You will insert it here to show how unsupported operators can impact Ethos-U delegation.

Before attempting to lower a model with some additional unsupported operators, try to predict what might happen:

• Will the model still have Ethos-U delegated regions?
• Will the graph remain one large delegate or split into multiple delegates?
• Which evidence would show that LRN introduced fallback work?
• What problem or problems could this cause?

Identify the output that would confirm your prediction. Consider also any potential impacts of your prediction on performance.

Insert LocalResponseNorm into MobileNetV2 before quantization and lowering:

    

        
        
python module-4-ethos-npu-accelerated-inference/scripts/aot/quantize_lower_ethosu.py \
  --spec module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/model_spec.json \
  --calibration-images module-4-ethos-npu-accelerated-inference/artifacts/mobilenetv2/calibration_images \
  --num-calib 100 \
  --target ethos-u85-256 \
  --system-config Ethos_U85_SYS_DRAM_Mid \
  --memory-mode Shared_Sram \
  --insert-lrn

    

The wrapper used by the script adds LRN after the MobileNetV2 feature extractor:

    

        
        
def forward(self, image):
    x = self.features(image)
    x = self.lrn(x)
    x = F.adaptive_avg_pool2d(x, (1, 1))
    x = torch.flatten(x, 1)
    return self.classifier(x)

    

The script prints the top-level ExecuTorch delegation summary. You should expect output similar to this:

    

        
        
PTE size: 3.60 MB

Delegation
==========
Ethos-U delegate call sites: 2
Fallback call_function ops:  13
  aten.avg_pool3d.out (1)
  cortex_m.dequantize_per_tensor.out (2)
  cortex_m.quantize_per_tensor.out (2)
  ...

    

The supported operations on either side of the LRN path still compile through Vela, but aten.avg_pool3d.out remains non-delegated between two NPU-delegated regions. Vela therefore generates two Ethos-U command streams instead of one. The cortex_m.quantize_per_tensor and cortex_m.dequantize_per_tensor entries are the boundary q/dq nodes that the script rewrites before packaging. The full output also contains getitem and alloc entries. These small operations select outputs and allocate memory.

Compare the non-LRN Vela report with the two LRN-region Vela reports:

These are per-region compiler estimates, not measured latency. The reports show that the LRN variant is split into two compilable NPU regions and that its estimated memory traffic differs substantially from the baseline.

A single-delegate pattern looks like:

    

        
        
ExecuTorch runtime -> [large Ethos-U delegated region] -> ExecuTorch runtime

    

The LRN variant looks closer to:

    

        
        
ExecuTorch runtime -> [Ethos-U region] -> [q/dq + non-delegated avg_pool3d] -> [Ethos-U region] -> ExecuTorch runtime

    

The second pattern still uses the NPU, but moving between two delegates can add q/dq operations, synchronization, and tensor movement.

Whether the non-delegated operation can execute on a target CPU also depends on the kernels included in that target runtime, and even if it can execute, it could add a lot of additional overhead, bottlenecking your application, and potentially obscuring benefits gained through NPU acceleration.

Before moving on to running the original INT8 model on the FVP, you may wish to use Model Explorer to visualize the different artifacts.

Optional activity