Understand the dual-NPU architecture

What models you’ll run

The Alif Ensemble E8 combines two Arm Cortex-M55 cores with two different Ethos-U NPUs. You’ll run the workloads on the high-performance Cortex-M55 core. One model is assigned to each neural processing unit (NPU):

How the application reduces the system power cost

Power-constrained embedded products often need more than one machine learning workload, but adding another microcontroller unit (MCU) also adds its active and idle power. Adding another MCU can also add external memory, interprocessor communication, and board-level power domains. The E8 avoids that duplication by letting one Cortex-M55 coordinate both on-chip NPUs. Camera capture, preprocessing, result fusion, and the user interface stay on one MCU, while each NPU runs the workload it handles best.

This arrangement doesn’t make two active NPUs consume less instantaneous power than one active NPU. It reduces system-level overhead compared with a design that needs a separate MCU or application processor for each NPU. It can also finish concurrent workloads sooner, allowing the system to return to an idle or lower-power state earlier. Measure energy on the final hardware to quantify the saving for a specific duty cycle.

The application uses the native Zephyr drivers in the Alif SDK main branch. The MT9M114, image signal processor (ISP), and MW405 changes were merged through pull request 879. The following diagram extends the original multi-NPU prototype with the live camera, ISP, and display pipeline used by this demo.

Dual-NPU live vision data flow

How the application separates the hardware responsibilities

One Zephyr application owns two driver objects, two register windows, two interrupt handlers, and two timing records:

The two interrupt service routines call ethosu_irq_handler() with the matching driver. A U85 completion therefore can’t release the U55 wait object. A U55 completion can’t change the U85 timing record.

How the application selects the NPU from model metadata

The application enables the Ethos-U core driver’s multi-variant support with ETHOSU_MULTI_VARIANT. This support is part of the core-driver main branch. During initialization, ethosu_init_ex() registers both physical devices with their product descriptors.

The motivation for this multi-variant driver support is to let one MCU manage different Ethos-U products in the same system. Without multi-variant driver support, software integration tends toward separate driver instances or separate processing domains for each NPU variant. A common registry keeps device discovery, interrupt handling, and workload dispatch in one Zephyr application. This supports the lower-overhead system architecture described earlier.

Vela stores a COP1 optimizer record in each delegated program. The record identifies the target product and MAC configuration:

The ExecuTorch backend parses this record and asks the registry for a compatible free driver. Device selection is therefore a property of the compiled PTE model rather than a hard-coded assumption in the worker thread.

How the application processes each frame

The camera and display use different MIPI D-PHY instances so they can operate at the same time. The J16 MT9M114 camera uses D-PHY 0, while the MW405 display uses D-PHY 1.

For every live frame, the application performs the following steps:

1. The MT9M114 sends 1288 x 728 RAW10 data over MIPI CSI-2.
2. The hardware ISP crops and demosaics the image into a 192 x 192 planar RGB888 buffer.
3. Zephyr maintains five video buffers so capture continues while one frame is processed.
4. The coordinator creates a 120 x 160 grayscale SSD tensor and a 224 x 224 RGB MobileNetV2 tensor.
5. The coordinator releases both persistent worker threads through separate semaphores.
6. Each worker copies its input into its prepared ExecuTorch method and submits the delegated command stream.
7. The coordinator waits for both completion semaphores, records timing, and returns the captured buffer to the ISP queue.
8. The UI compositor draws a 480 x 352 RGB565 preview, face boxes, the classification result, and rolling timing values in the 480 x 800 framebuffer.

ExecuTorch program and method construction is serialized because that setup path contains shared runtime state. After setup, each worker retains an independent immutable Method, allocator set, and NPU backend. Only prepared model execution runs in parallel. Camera capture, preprocessing, result fusion, and display updates remain coordinated by the Cortex-M55.

How the application coordinates the workers

The coordinator uses three synchronization stages:

1. Each worker signals that its Program and Method are prepared.
2. The coordinator gives both execute semaphores for the current frame.
3. Each worker signals after it has copied the input and again after inference completes.

The early input-copy signal lets the coordinator return the camera buffer to the ISP queue without waiting for both NPUs. The later completion signal protects result processing and timing calculations.

The application records execution start and end cycles inside each worker, close to Method::execute(). It derives four live metrics:

These measurements isolate delegated execution from camera capture and UART output. They don’t represent complete camera-to-display latency.

Understand the memory layout

The application separates persistent artifacts, CPU-private state, display and video buffers, and U85-visible working memory.

Memory placement for the dual-NPU application

Persistent MRAM payload

SETOOLS writes model_assets.bin at 0x80008000 and the execute-in-place Zephyr image at 0x80400000:

The combined model_assets.bin payload is 3,896,318 bytes. CMake packs both PTE files first, followed by the startup image and the labels. CMake then generates a header containing the artifact sizes. Changing a model causes CMake to reconfigure so the compiled offsets can’t silently disagree with the payload.

HP DTCM

The 1 MiB HP DTCM region starts at 0x20000000. It contains the following:

• 640 KiB U55 method pool
• Both 4 KiB metadata pools
• The fast-scratch arrays
• Zephyr worker stacks
• Semaphores
• Driver objects
• ISP library state

The larger U55 temporary arena is placed in SRAM0. The linker can move individual DTCM symbols as code changes, so the application fixes their sizes rather than their exact addresses.

Shared SRAM0

SRAM0 spans from 0x02000000 to 0x023FFFFF. The validated build dedicates most of it to the U55 delegate scratch arena:

SSD-Slim requires a larger delegated scratch plan than the earlier face detector. Keeping this arena outside HP DTCM leaves enough private memory for the method pool and Zephyr runtime.

Shared SRAM1

SRAM1 spans from 0x02400000 to 0x027FFFFF and holds the display, U85-visible working set, and camera buffers:

The active ISP output uses five 192 x 192 x 3-byte RGB888 buffers. Each buffer is 110,592 bytes. The heap allows up to 114,688 bytes per buffer. When its original address isn’t directly usable by U85, the backend copies delegated command or weight data into the mirror reservation. Shared SRAM also avoids consuming the limited HP DTCM with the U85 working set.

How the application maintains cache and address visibility

The Cortex-M55 data cache isn’t coherent with the camera, display controller, or either NPU. The application follows two rules:

• Clean or flush CPU-written input tensors and framebuffer pixels before a device reads them.
• Invalidate captured frames and NPU-written output tensors before the CPU reads them.

The U55 and U85 also require system-visible addresses. The platform layer translates CPU-local addresses to their Alif global aliases and selects the correct AXI memory attributes for MRAM, HP DTCM, SRAM0, and SRAM1. Incorrect address translation or memory attributes can produce an NPU bus error even when the CPU can read the same bytes.

What you’ve learned and what’s next

You’ve learned how the application’s dual-NPU architecture works and understood the application’s memory layout.

Next, you’ll prepare the board and a clean west workspace for this architecture.