GPT-2 and DistilBERT runtime results
Compare benchmark and graph results
You should now have three kinds of evidence: benchmark timings, Model Explorer observations, and the behavior of the two Pi demos. For each row below, record the result you observed and one short conclusion it supports. This will help you answer the assessment questions that follow.
| Investigation area | GPT-2 / Decoder Transformer | DistilBERT / Encoder Transformer | What it helps you decide |
|---|---|---|---|
| Average PyTorch latency | Whether ExecuTorch can provide better performance than the full PyTorch stack. | ||
| Portable ExecuTorch latency | Whether the portable ExecuTorch deployment path is fast enough without backend optimization. | ||
| XNNPACK delegated latency | Whether ExecuTorch with an optimized backend is a practical CPU deployment option. | ||
| Relative speedup after delegation | Whether backend delegation provides a meaningful performance benefit for this workload. | ||
| Delegate call sites / graph fragmentation | Whether the graph contains a few large delegated regions or many smaller regions separated by fallback work. | ||
| Remaining fallback execution observations | Whether unsupported or non-delegated operations are limiting performance gains. | ||
| Exported .pte observations for portable and XNNPACK | Whether using XNNPACK increases the .pte size and what the implication is. | ||
| Observed application behavior | Whether the exported and delegated model still functions correctly for its intended task. |
Entries are saved in this browser.
Course complete
Nice work
You completed Optimize and deploy AI inference with ExecuTorch on Arm.