Choose a runtime path with PyTorch, ExecuTorch, and XNNPACK

The main question at the outset of this module was “When should you use ExecuTorch instead of the full PyTorch stack on an Arm CPU, and what is the optimal way to do so?"

The following table summarizes likely choices:

Situation	Likely choice	Why
You are developing, debugging, or experimenting on a powerful system	PyTorch	Maximum flexibility and familiar tooling
The edge target can support the full PyTorch stack and latency is acceptable	PyTorch may be sufficient	Deployment constraints are not forcing a lighter runtime
The edge target cannot support full PyTorch because of footprint or dependencies	ExecuTorch with portable kernels or a supported backend	Smaller runtime and dependency footprint than full PyTorch
The target is CPU-only, latency matters, and the model delegates well	ExecuTorch with XNNPACK	Optimized backend kernels can improve supported operations
The workload delegates poorly and does not meet a strict latency target	Reconsider the model, backend, or target	Runtime choice cannot rescue a poor model-task-device fit

PyTorch eager execution is the baseline. If the full PyTorch stack is acceptable on your target and meets the latency target, it may be a reasonable choice.

Portable ExecuTorch kernels provide broad operator coverage and a compact deployment path, but they are not usually the fastest option. They are useful when fitting the full PyTorch stack on the target is the main constraint and functional execution matters more than latency.

XNNPACK performance depends on the work inside its delegated partitions, not only the number of delegate or fallback calls. Large, compute-heavy partitions can provide a substantial benefit. Fallback operators may perform inexpensive allocation, indexing, shape handling, or mask processing, while frequent transitions between delegated and non-delegated execution can add overhead.

Acceptable latency depends on the application and target hardware. Module 2 will show how quantization can further reduce model size and latency.

Choose a runtime path with PyTorch, ExecuTorch, and XNNPACK

Nice work