Developer Hub

Learning Paths

Courses

Optimize and deploy AI inference with ExecuTorch on Arm

...

End of course survey

Check your understanding

Course progress

0% complete

0 Introduction and setup

Introduction

About this course How to use this resource

1 Compare Transformer Inference on Arm CPUs

1.1Introduction

About this module What will you be able to do?

1.2Before you begin

Understand the runtime paths you will compare

1.3Plan the ExecuTorch deployment workflow

Follow the workflow from model export to application testing Development machine setup

1.4Evaluate a decoder-based transformer deployment

Export, lower, and benchmark GPT-2 with ExecuTorch (Portable and XNNPACK) GPT-2 preparation and export GPT-2 lowering Inspect GPT-2 delegation with Model Explorer Raspberry Pi setup Copy GPT-2 artifacts to Raspberry Pi GPT-2 benchmark on Raspberry Pi View the GPT-2 benchmark results Run the GPT-2 text-generation demo

1.5Evaluate an encoder-based transformer deployment

Export, lower, and benchmark DistilBERT with ExecuTorch and XNNPACK DistilBERT preparation and export DistilBERT lowering Inspect DistilBERT delegation with Model Explorer Copy DistilBERT artifacts to Raspberry Pi DistilBERT benchmark on Raspberry Pi View the DistilBERT benchmark results Run the DistilBERT autocomplete demo

1.6Compare deployment trade-offs from the workflow

GPT-2 and DistilBERT runtime results Choose a runtime path with PyTorch, ExecuTorch, and XNNPACK

1.7Check your understanding

Review key module learnings

2 CNN inference on Arm Cortex-A and optimize with INT8 quantization

2.1Introduction

About this module What will you be able to do? Why MobileNetV2 is appropriate for edge CPU inference

2.2Plan and prepare MobileNetV2

MobileNetV2 deployment workflow Establish the shared model inputs

2.3Export and lower MobileNetV2 with ExecuTorch and XNNPACK

Create the FP32 XNNPACK artifact Understand export and lowering Inspect the delegation summary

2.4Benchmark PyTorch and FP32 ExecuTorch on the Raspberry Pi

Measure on target hardware Interpret latency and classification output

2.5Quantize MobileNetV2 to INT8 with XNNPACK

Convert the FP32 workflow into an INT8 deployment path Perform Quantization

2.6Evaluate INT8 MobileNetV2

Benchmark the INT8 artifact on Raspberry Pi Compare MobileNetV2 deployment results Use the model in an application

2.7Check your understanding

Review the key delegation and quantization principles

2.8Optional: Run an Object Detection application

Deploy a pre-optimized YOLOv8s object detector from Arm AI Portal

2.9Optional: Profile ExecuTorch runtime events

Inspect runtime events with ETDump

3 Deploy INT8 semantic segmentation in an Android app

3.1Introduction

About this module What you will learn ExecuTorch deployment on Android What is semantic segmentation

3.2Plan and prepare Android segmentation

Android segmentation deployment workflow Development machine setup Establish the model and output specification

3.3Create and validate the FP32 model

Create the local quality baseline Does the model function correctly for the use-case?

3.4Quantize LR-ASPP to INT8 and compare cutout quality

Measure the size and quality trade-off Compare FP32 and INT8 cutouts

3.5Build and run the Android application

Prepare the Android build tools Run the INT8 model in an Android app

3.6Inspect the Android runtime code

Model vs application responsibilities Load the model asset Run ExecuTorch and validate the output

3.7Check your understanding

Check your understanding

4 Deploy MobileNetV2 with ExecuTorch and Arm Ethos-U acceleration

4.1Introduction

About this module What you will learn What is an Arm Ethos-U NPU?

4.2Prepare for Ethos-U development

Ethos-U deployment workflow Ubuntu machine setup

4.3Compare MobileNetV2 delegation outcomes

Test FP32 Ethos-U delegation Create the INT8 Ethos-U artifact How operator support changes delegation

4.4Run MobileNetV2 on Corstone-320 FVP

Validate the target execution path Create the FVP output Build and run the executor runner Decode the output

4.5Build a MobileNetV2 classification app

Add application-specific output handling Build the classification app runner

4.6Review the Ethos-U deployment

Review the Ethos-U Deployment

4.7Check your understanding

Review the key Ethos-U acceleration principles End of course survey

Next steps

Further learning

Continue your ExecuTorch learning

End of course survey

For the best experience on this narrow screen, open the survey in a new tab.

Open survey in a new tab

Previous page

Complete and continue

Course complete

Nice work

You completed Optimize and deploy AI inference with ExecuTorch on Arm.

Explore next steps