Developer Hub

Learning Paths

Courses

Optimize and deploy AI inference with ExecuTorch on Arm

...

Review the key Ethos-U acceleration principles

Check your understanding

Course progress

0% complete

0 Introduction and setup

Introduction

About this course How to use this resource

1 Compare Transformer Inference on Arm CPUs

1.1Introduction

About this module What will you be able to do?

1.2Before you begin

Understand the runtime paths you will compare

1.3Plan the ExecuTorch deployment workflow

Follow the workflow from model export to application testing Development machine setup

1.4Evaluate a decoder-based transformer deployment

Export, lower, and benchmark GPT-2 with ExecuTorch (Portable and XNNPACK) GPT-2 preparation and export GPT-2 lowering Inspect GPT-2 delegation with Model Explorer Raspberry Pi setup Copy GPT-2 artifacts to Raspberry Pi GPT-2 benchmark on Raspberry Pi View the GPT-2 benchmark results Run the GPT-2 text-generation demo

1.5Evaluate an encoder-based transformer deployment

Export, lower, and benchmark DistilBERT with ExecuTorch and XNNPACK DistilBERT preparation and export DistilBERT lowering Inspect DistilBERT delegation with Model Explorer Copy DistilBERT artifacts to Raspberry Pi DistilBERT benchmark on Raspberry Pi View the DistilBERT benchmark results Run the DistilBERT autocomplete demo

1.6Compare deployment trade-offs from the workflow

GPT-2 and DistilBERT runtime results Choose a runtime path with PyTorch, ExecuTorch, and XNNPACK

1.7Check your understanding

Review key module learnings

2 CNN inference on Arm Cortex-A and optimize with INT8 quantization

2.1Introduction

About this module What will you be able to do? Why MobileNetV2 is appropriate for edge CPU inference

2.2Plan and prepare MobileNetV2

MobileNetV2 deployment workflow Establish the shared model inputs

2.3Export and lower MobileNetV2 with ExecuTorch and XNNPACK

Create the FP32 XNNPACK artifact Understand export and lowering Inspect the delegation summary

2.4Benchmark PyTorch and FP32 ExecuTorch on the Raspberry Pi

Measure on target hardware Interpret latency and classification output

2.5Quantize MobileNetV2 to INT8 with XNNPACK

Convert the FP32 workflow into an INT8 deployment path Perform Quantization

2.6Evaluate INT8 MobileNetV2

Benchmark the INT8 artifact on Raspberry Pi Compare MobileNetV2 deployment results Use the model in an application

2.7Check your understanding

Review the key delegation and quantization principles

2.8Optional: Run an Object Detection application

Deploy a pre-optimized YOLOv8s object detector from Arm AI Portal

2.9Optional: Profile ExecuTorch runtime events

Inspect runtime events with ETDump

3 Deploy INT8 semantic segmentation in an Android app

3.1Introduction

About this module What you will learn ExecuTorch deployment on Android What is semantic segmentation

3.2Plan and prepare Android segmentation

Android segmentation deployment workflow Development machine setup Establish the model and output specification

3.3Create and validate the FP32 model

Create the local quality baseline Does the model function correctly for the use-case?

3.4Quantize LR-ASPP to INT8 and compare cutout quality

Measure the size and quality trade-off Compare FP32 and INT8 cutouts

3.5Build and run the Android application

Prepare the Android build tools Run the INT8 model in an Android app

3.6Inspect the Android runtime code

Model vs application responsibilities Load the model asset Run ExecuTorch and validate the output

3.7Check your understanding

Check your understanding

4 Deploy MobileNetV2 with ExecuTorch and Arm Ethos-U acceleration

4.1Introduction

About this module What you will learn What is an Arm Ethos-U NPU?

4.2Prepare for Ethos-U development

Ethos-U deployment workflow Ubuntu machine setup

4.3Compare MobileNetV2 delegation outcomes

Test FP32 Ethos-U delegation Create the INT8 Ethos-U artifact How operator support changes delegation

4.4Run MobileNetV2 on Corstone-320 FVP

Validate the target execution path Create the FVP output Build and run the executor runner Decode the output

4.5Build a MobileNetV2 classification app

Add application-specific output handling Build the classification app runner

4.6Review the Ethos-U deployment

Review the Ethos-U Deployment

4.7Check your understanding

Review the key Ethos-U acceleration principles End of course survey

Next steps

Further learning

Continue your ExecuTorch learning

Review the key Ethos-U acceleration principles

Question 1: Which recorded result would most strongly indicate that a model is well suited for Ethos-U acceleration?

Successful FP32 eager execution on the CPU Successful FP32 CPU execution shows the model can run, but it does not show Ethos-U suitability. High delegation coverage with minimal non-delegated work This shows that most of the graph can execute on the accelerator without costly fallback work. Large .pte artifact size after export Artifact size is a deployment signal, but it does not prove the model delegated to the NPU. High Python execution throughput during calibration Calibration throughput happens on the host and does not show target-side accelerator suitability.

Check answer

Retry

Question 2: What does successful Corstone FVP execution show?

Final real-hardware latency and power The FVP does not provide final real-board latency or power. Those require measurement on representative hardware. That the generated deployment flow can execute in a target-like software environment The FVP validates the software path and generated artifact before testing on physical hardware. That all future inputs will preserve model accuracy A successful run on one sample image does not prove that all future inputs preserve model quality. That all non-delegated work has been eliminated The FVP run shows that the delegated path executed, but it does not prove that all non-delegated work has been eliminated.

Check answer

Retry

Previous page

Complete and continue

Answer every question correctly to complete this page.

Course complete

Nice work

You completed Optimize and deploy AI inference with ExecuTorch on Arm.

Explore next steps