ExecuTorch for Cortex-M and Ethos-U

ExecuTorch is the PyTorch runtime for deploying machine learning models on edge devices, including embedded systems. It converts trained models into programs that can run within the limited memory and compute resources of a microcontroller.

Many microcontrollers use Arm Cortex-M CPUs. Some also integrate an Arm Ethos-U NPU to accelerate neural-network operations.

For combined Cortex-M and Ethos-U systems, ExecuTorch lets one model use both processors. Supported operations are dispatched to the Ethos-U NPU, while remaining operations run on the Cortex-M CPU.

Choose where each operation runs

The example model uses three execution paths:

Execution pathWhat operation runs thereWhy the operation runs there
Ethos-U55 NPU The stride-1 convolutionThe NPU supports this convolution configuration.
CMSIS-NN kernels on Cortex-M The stride-4 convolutionEthos-U55 doesn’t support this configuration, so the CPU runs it with an optimized CMSIS-NN kernel.
ExecuTorch portable kernels on Cortex-MThe topk operationThis operation uses a general CPU implementation.

The following diagram shows the NPU path alongside the two paths that run on the Cortex-M CPU:

Image Alt Text:Diagram of the simulated Corstone-300 target showing the Cortex-M CPU dispatching the stride-1 convolution to Ethos-U55, running the stride-4 convolution with CMSIS-NN, and running topk with a portable ExecuTorch kernelExecuTorch execution paths

Fallback is expected here during export. It isn’t recovery from an NPU failure at runtime. The exporter determines which regions the target Ethos-U delegate supports. It then assigns known remaining patterns to the Cortex-M backend.

Note

Ethos-U55 is used for this Learning Path because its unsupported stride-4 convolution creates a predictable fallback region. Ethos-U85 supports more operator configurations, so it will typically have less CPU fallback than Ethos-U55. The same workflow still applies when a model contains operations that remain outside the U85 delegate.

You’ll export a model and select CMSIS-NN kernels for its Cortex-M operations. You’ll then run a mixed CPU and NPU application on a Corstone-300 Fixed Virtual Platform (FVP). The checks show which implementations the model uses and confirm that inference completes. Measure latency and power on your target hardware to quantify the benefit for your application.

The CMSIS-Executorch example that you’ll clone already contains the necessary board support, runner, memory configuration, and FVP settings.

Check the host tools

You need Visual Studio Code (VS Code) for the Keil Studio workflow. Before cloning the project, check that Git and Python 3.10–3.13 are available in your host terminal:

    

        
        

git --version
python3 --version
  

    
    

        
        

git --version
python --version
  

    

If either command is unavailable, install Git or a supported Python version before continuing.

Note

Python 3.14 is excluded because the pinned Tensor Operator Set Architecture (TOSA) export dependency has no wheel for it.

On Linux, ensure that your Python installation includes venv support. Install the python3-venv package for your distribution if needed.

Note

The macOS exporter requires Apple silicon and macOS 15 or later. The pinned ExecuTorch and TOSA packages don’t provide Intel macOS wheels. Docker supplies the FVP on macOS, and the Python exporter runs on the host.

On Windows, enable long-path support before installing the export dependencies.

Clone the example project

Clone the CMSIS-Executorch example and select the tested revision:

    

        
        
git clone https://github.com/Arm-Examples/CMSIS-Executorch.git
cd CMSIS-Executorch
git checkout 4002b63dcc3e27eb7444946a0dbf69b9a74d6fc9

    

The CMSIS-Executorch directory is the project root. Open this folder in VS Code and use it as the working directory for every command.

Install the VS Code extensions

In the VS Code Extensions view, install the Keil Studio Pack and the Python extension . Keil Studio Pack includes Arm Tools Environment Manager and the CMSIS Solution extension that you’ll use.

Image Alt Text:VS Code extension details for Arm Keil Studio Pack. The Disable and Uninstall buttons confirm that the extension pack is installed.Keil Studio Pack installed in VS Code

Activate the Arm tools and license

When you open the project folder, Arm Tools Environment Manager automatically activates vcpkg-configuration.json by default. If prompted, allow it to download the declared tools. Wait for the Arm Tools status-bar indicator to finish loading. Hover over it to check the installed tools, or open Arm Tools > View Log to inspect activation errors.

Image Alt Text:VS Code Arm Tools status bar showing seven activated tools. The tooltip lists CMSIS-Toolbox 2.15.0 and Arm Compiler 6.24.0 among the tools added to the integrated terminal’s PATH.Confirm that the project's Arm tools are active

If activation doesn’t start, right-click vcpkg-configuration.json in the Explorer view and select Activate Environment. You can also select Activate Environment from the Arm Tools status-bar menu when the configuration is detected.

Arm Compiler for Embedded requires an activated license. Select the license indicator next to Arm Tools, then select Activate Arm Keil MDK Community Edition for evaluation or non-commercial use. Alternatively, you can select Activate or manage Arm licenses to use your existing license.

Confirm that the indicator shows an active license before building. For available options, see Keil Studio license activation .

In the CMSIS view, select … > Open Solution in Workspace and choose cmsis-executorch-u55.csolution.yml. Confirm that the active target is SSE-300-U55.

Image Alt Text:VS Code Manage Solution Target screen for cmsis-executorch-u55.csolution.yml. SSE-300-U55 is selected and the cmsis-executorch project uses the Debug build type.Select the Corstone-300 Ethos-U55 target

Create the Python environment

Select Terminal > Run Task > Create Python environment (pip).

Image Alt Text:VS Code task picker with Create Python environment (pip) selected. Run this task to create .venv and install the model-export dependencies.Run the pip environment setup task

The task runs setup_venv.sh on Linux and macOS, or setup_venv.bat on Windows. Both invoke setup_venv.py. The task completes the following setup steps:

  • Creates .venv using the host Python interpreter.
  • Installs requirements.txt, including ExecuTorch 1.5.1, PyTorch 2.14.0, TorchAO 0.18.0, and Vela 5.1.0.
  • Installs requirements-arm-tosa.txt with --no-deps to preserve the required serialization versions of the exporter.
  • Checks that the ExecuTorch Arm quantizer imports successfully.

Wait for the task to exit successfully and print a message similar to:

    

        
        venv ready: .../CMSIS-Executorch/.venv

        
    

This task installs the Python dependencies for you. Use the setup script when repairing the environment so that both requirements files are installed with the correct options.

If the task selects a different Python version from the supported one that you installed, run the setup from the project root. For example, with Python 3.12 installed:

    

        
        

PYTHON=python3.12 ./setup_venv.sh
  

    
    

        
        

py -3.12 setup_venv.py
  

    

Open and verify the working terminal

After Arm tool activation and Python setup, select Terminal > New Terminal.

Use a Bash-compatible shell on Linux and macOS, or PowerShell on Windows. This new integrated terminal receives the activated Arm tool environment.

Activate .venv in this terminal:

    

        
        

source .venv/bin/activate
  

    
    

        
        

.\.venv\Scripts\Activate.ps1
  

    

Check the working directory and Python interpreter:

    

        
        
python -c "from pathlib import Path; import sys; print('Project:', Path.cwd()); print('Python:', sys.executable)"

    

The output is similar to:

    

        
        Project: .../CMSIS-Executorch
Python: .../CMSIS-Executorch/.venv/bin/python

        
    

On Windows, the interpreter path ends in .venv\Scripts\python.exe. Both paths need to belong to the same project. Keep using this terminal for subsequent commands. Repeat environment activation and this check if you need to open a new terminal.

Verify the environment

Check the activated Arm tools and Python package versions:

    

        
        
python --version
cbuild --version
armclang --version
python -c "from importlib.metadata import version; print('ExecuTorch', version('executorch')); print('PyTorch', version('torch'))"

    

Confirm that Python is between 3.10 and 3.13, cbuild reports 2.15.0 or later, and Arm Compiler for Embedded reports 6.24.

The expected final two lines of the output are:

    

        
        ExecuTorch 1.5.1
PyTorch 2.14.0

        
    
Note

If a tool is unavailable or its version differs, reactivate the Arm tools and open a new integrated terminal. Activate .venv again before repeating these checks. The ExecuTorch Python package and CMSIS Pack versions need to match because the embedded runtime loads the program produced by the exporter.

Set up Docker on macOS

On macOS, install and start Docker Desktop . The FVP runs inside a Linux container because it isn’t available as a macOS application.

Check that Docker is running:

    

        
        
docker info

    

The project .vscode/fvp.sh wrapper mounts the project directory into the container, including when it’s outside your home directory. Its first run builds the FVP container image.

Download the Learning Path assets

Download the example folder for the Learning Path from the code examples repository. The archive is pinned to a commit and includes the fallback model, integration patch, inspection helpers, and their license. Run the commands from the project root. They create lp-assets for the downloaded originals, then copy the four working files into the project root:

    

        
        

curl -fL "https://gitlab.arm.com/learning-code-examples/code-examples/-/archive/5eb4e50676d4085c90e926abec1c978f257d1bde/code-examples-5eb4e50676d4085c90e926abec1c978f257d1bde.tar.gz?path=learning-paths/embedded-and-microcontrollers/optimize-executorch-fallback-cortex-m" -o lp-assets.tar.gz
mkdir -p lp-assets
tar -xzf lp-assets.tar.gz -C lp-assets --strip-components=4
cp lp-assets/*.py lp-assets/*.patch .
  

    
    

        
        

curl.exe -fL "https://gitlab.arm.com/learning-code-examples/code-examples/-/archive/5eb4e50676d4085c90e926abec1c978f257d1bde/code-examples-5eb4e50676d4085c90e926abec1c978f257d1bde.tar.gz?path=learning-paths/embedded-and-microcontrollers/optimize-executorch-fallback-cortex-m" -o lp-assets.tar.gz
New-Item -ItemType Directory -Force lp-assets | Out-Null
tar -xzf lp-assets.tar.gz -C lp-assets --strip-components=4
Copy-Item lp-assets\*.py, lp-assets\*.patch .
  

    

Keep track of the project files

The working files now have the following roles:

    

        
        
CMSIS-Executorch/
├── lp-assets/                       Downloaded originals and license
├── mixed_backend_cnn.py             Model to copy into model/model.py
├── export_cortex_m_fallback.py       Export and inspect both graph variants
├── enable_cortex_m_fallback.patch   Add CMSIS-NN fallback to the project exporter
├── verify_pack_sources.py           Check the build's Pack source paths
├── model/model.py                   Working model used by both exporters
├── .venv/                           Python export environment
├── create_ai_layer.py               Project exporter, patched in chapter 3
├── cmsis-executorch-u55.csolution.yml
├── artifacts/                       Inspection exports from chapters 2 and 3
├── ai_layer/                        Generated model and firmware component list
└── out/                             Compiled firmware and compilation database

    

You’ll create the artifacts/ and out/ directories later. ai_layer/ already exists in the cloned example and will be regenerated for your model.

The inspection script writes its .pte and graph-summary .json files into artifacts/. The firmware embeds ai_layer/model.pte, which create_ai_layer.py generates. Keep all commands rooted in CMSIS-Executorch so that these paths continue to refer to the same files.

Use a convolution example

The CMSIS-Executorch repository starts with a TinyCNN model that can be delegated completely to Ethos-U.

You’ll change this model to one that will demonstrate fallback. You’ll retain the model input and output interface, but adjust the operations happening inside it.

Replace the original model with the one that you downloaded:

    

        
        

cp mixed_backend_cnn.py model/model.py
  

    
    

        
        

Copy-Item .\mixed_backend_cnn.py .\model\model.py
  

    

The model still accepts one 16 x 16 RGB image in NHWC order and returns ten values. Keeping that interface means that the supplied bare-metal runner needs no changes.

The important part of model/model.py is now as follows:

    

        
        
self.conv1 = nn.Conv2d(
    channels, 4, kernel_size=3, stride=1, padding=1, bias=False
)
self.conv2 = nn.Conv2d(
    4, num_classes, kernel_size=3, stride=4, padding=1, bias=False
)

values = torch.relu(self.conv1(values))
values = self.conv2(values)
values = torch.flatten(values, start_dim=2)
values, _ = torch.topk(values, k=1, dim=-1)

    

The stride controls how far a convolution filter moves between positions. stride=1 visits adjacent positions. stride=4 moves four positions at a time and produces a smaller output. For this Ethos-U55 target and tensor shape, the first convolution can be delegated to the NPU. The stride-4 convolution remains outside the delegate. topk selects the largest spatial response for each of the ten output channels.

What you’ve accomplished and what’s next

You now have a pinned Corstone-300 Ethos-U55 project, and its matching Python packages and versioned Arm tools. You’ve also downloaded a model that will run only partially on an Ethos-U55.

Next, you’ll understand how to optimize supported operators with CMSIS-NN on Cortex-M CPUs in a CPU-only example.