About this module

In this module, you will compare three ways to run transformer inference on an Arm CPU:

• PyTorch eager execution

• ExecuTorch portable runtime

• ExecuTorch with XNNPACK backend

You will work through a practical ExecuTorch deployment workflow and compare the trade-offs that appear along the way: runtime choice, artifact size, backend delegation, target-device latency, and application behavior.

You will use ExecuTorch’s ‘ahead-of-time’ (AOT) workflow to prepare models on a development machine, produce a delegation summary, and then run the models on a Raspberry Pi to collect benchmark results. For each model, the workflow produces two .pte artifacts, a delegation summary, target benchmark results, a working demo, and a runtime recommendation. By the end, you should be able to explain when ExecuTorch is a better deployment choice than the full PyTorch stack. You will use two transformer models:

GPT-2: a decoder-only transformer used for text generation

DistilBERT: an encoder-only transformer used for prediction-style NLP tasks