Run parallel vision inference on an Alif Ensemble E8 with Zephyr
Introduction
Understand the dual-NPU architecture
Prepare the board and workspace
Build the dual-NPU application
Package and flash the application
Validate live parallel inference
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

Connect the target hardware

Power off the E8 DevKit before changing camera or display connections. Then, to connect the hardware:

  1. Connect the MT9M114 camera module to the bottom-side J16 connector.
  2. Connect the MW405 display to the display connector.
  3. Connect the board’s USB ports for power, SE UART, and U4 UART.
  4. Confirm that the board runs SEROM 1.105.65 and SERAM 1.110.0.
  5. Move the boot switch to the SE position before flashing.
Note
  • The supplied overlay targets the J16 selfie-camera connection. J22 uses a different I2C address and device-tree route. Don’t combine a J16 overlay with a camera connected to J22.
  • If the camera reports chip ID 0000 or I2C error -5, power off the board and check the camera connection. The supplied overlay expects the MT9M114 on J16 at the selfie-camera I2C address. Reseat the flex cable and confirm that its contacts face the correct direction.

Install the host tools

Confirm that the Xcode Command Line Tools are installed on your host machine:

    

        
        
xcode-select -p

If the command reports that the tools are missing, install them before you continue:

    

        
        
xcode-select --install

Install Git, CMake, and Python 3.12 with Homebrew. Then, create the west Python environment:

    

        
        
brew install git cmake python@3.12
mkdir -p $HOME/alif-dual-npu
cd $HOME/alif-dual-npu
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install \
  west==1.5.0 \
  pyelftools==0.33 \
  fdt==0.3.3 \
  ninja==1.13.2

Confirm that west and ninja are available:

    

        
        
west --version
ninja --version

If available, both commands print a version number.

Note

  • If CMake reports that it can’t find ninja, install it in the active virtual environment:

        

        
        
source $HOME/alif-dual-npu/.venv/bin/activate
python -m pip install ninja

    Run the build again with --pristine.

  • If the Alif flash runner can’t import fdt, activate the same environment and install the missing module:

        

        
        
source $HOME/alif-dual-npu/.venv/bin/activate
python -m pip install fdt

Create the west workspace

Clone the SDK fork that contains the dual-NPU sample at the validated revision, then initialize a local west workspace from that checkout. The fork’s main branch stays synchronized with the Alif SDK main branch.

The dual-NPU application is maintained separately on the dual-npu-main-integration branch, which also includes the support for MT9M114, image signal processor, and MW405:

    

        
        
cd $HOME/alif-dual-npu
source .venv/bin/activate
git clone --branch dual-npu-main-integration --single-branch \
  https://github.com/varunchariArm/sdk-alif.git sdk-alif
git -C sdk-alif checkout d194c62d41422ccae9355637d91c4344e0d55d24
west init -l sdk-alif
west config manifest.project-filter +executorch
west update --narrow
python -m pip install -r zephyr/scripts/requirements.txt
west sdk install --toolchains arm-zephyr-eabi

The manifest project appears at sdk-alif. The remaining projects appear under modules, bootloader, tools, and zephyr.

Note

Don’t initialize from alifsemi/sdk-alif directly. The dual-NPU application hasn’t yet been merged there. Use the fork’s dual-npu-main-integration branch which contains the application.

Initialize the ExecuTorch submodules:

    

        
        
git -C modules/lib/executorch submodule update --init --recursive

Add the multi-variant dependencies

You’ll use the multi-variant support merged into the Ethos-U core driver main branch. This support allows one Cortex-M55 to manage the U55 and U85 through one driver registry, avoiding the system power overhead of assigning each NPU to a separate microcontroller unit.

Clone the current main branch:

    

        
        
git clone --branch main \
  https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-core-driver.git \
  modules/ethos-u-core-driver-src
git -C modules/ethos-u-core-driver-src merge-base --is-ancestor \
  b7cd193afde80afe8bbae9a26d2ca6586554f054 HEAD

The Alif west manifest also downloads Zephyr’s hal_ethos_u module. That module is a separately maintained snapshot. Its manifest revision doesn’t yet contain the merged multi-variant implementation. The explicit clone therefore remains necessary. The ancestor test is a guard rather than a pin: it permits newer main revisions while rejecting an old or stale checkout that can’t run U55 and U85 through the same driver registry.

Clone and pin CMSIS-NN:

    

        
        
git clone https://github.com/ARM-software/CMSIS-NN.git \
  modules/cmsis-nn-src
git -C modules/cmsis-nn-src checkout \
  d933672e7ca97eec70ef43230baee7b20c2a28ae

Create the Python environment that’s used by the ExecuTorch CMake integration:

    

        
        
python3.12 -m venv .venv-executorch
source .venv-executorch/bin/activate
python -m pip install --upgrade pip
python -m pip install \
  -r modules/lib/executorch/requirements-examples.txt
python -m pip install \
  west==1.5.0 \
  -r zephyr/scripts/requirements-base.txt
cd modules/lib/executorch
env -u DEBUG CMAKE_ARGS="-DEXECUTORCH_BUILD_MLX=OFF" \
  ./install_executorch.sh
cd ../../..
deactivate

Python 3.12 is used for compatibility with the pinned ExecuTorch revision.

Note

Removing a host DEBUG variable prevents ExecuTorch from interpreting a non-numeric shell value as its numeric build option. You don’t need the optional ethos_u Python dependency group for the firmware build.

Apply the sample’s ExecuTorch integration and Zephyr SRAM placement patches, then check the dependencies:

    

        
        
./sdk-alif/samples/modules/executorch/dual_npu_vision/setup_workspace.sh

The output is similar to:

    

        
        Applied ExecuTorch dual-NPU patch.
Applied Zephyr SRAM1 placement patch.
Ethos-U core driver main: ...
Workspace dependencies are ready.

If you run the script again, it reports that both patches are already applied. Run the script again after west update, which can restore either module checkout.

What you’ve accomplished and what’s next

You’ve now prepared the required sources and dependencies.

Next, you’ll build the dual-NPU application.

Back
Next