What is an Arm Ethos-U NPU?

An NPU is a specialized accelerator for the tensor operations used by neural networks. Convolutions and matrix multiplications contain large numbers of multiply-accumulate (MAC) operations. An NPU implements these operations with parallel data paths and memory access patterns designed for neural-network inference.

The Arm Ethos-U family targets embedded and edge systems with constrained compute, memory, power, and silicon area. Ethos-U configurations provide different numbers of MAC units. This module targets an Ethos-U85 with 256 MAC units.

The CPU remains part of the system. It starts ExecuTorch, configures memory, invokes delegated regions, and handles input, output, and application control flow. Any model operation outside the Ethos-U delegate needs a compatible CPU kernel in the target runtime.

As with XNNPACK, ExecuTorch partitions the graph and delegates supported regions. Work that is not delegated remains on the default ExecuTorch path. Unlike XNNPACK, Ethos-U is a separate processor rather than a collection of optimized CPU kernels.

Ethos-U executes quantized integer workloads rather than FP32 model compute. An FP32 model can still be exported and packaged as a .pte program, but no subgraphs in this example are delegated to Ethos-U. Its operations remain on the default ExecuTorch path and would require compatible Cortex-M kernels to run. This module does not execute the FP32 artifact.

Use an Ethos-U path when the model is quantized, and its important operators are supported. A CPU path may be the better choice when the model depends heavily on unsupported operations or when flexibility matters more than accelerator efficiency.