# Benchmark a KleidiAI micro-kernel in ExecuTorch

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/)
- [Set up your environment](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/01-env-setup/)
- [Cross-Compile ExecuTorch for the AArch64 platform](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/02-cross-compile/)
- [Accelerate ExecuTorch operators with KleidiAI micro-kernels](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/03-executorch-node-kai-kernel/)
- [Create and quantize linear layer benchmark model](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/04-create-fc-model/)
- [Create and quantize convolution layer benchmark model](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/05-create-conv2d-model/)
- [Create matrix multiply layer benchmark model](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/06-create-matrix-mul-model/)
- [Run model and generate the ETDump](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/07-run-model/)
- [Analyze ETRecord and ETDump](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/08-analyze-etdump/)
- [Next Steps](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/_next-steps/)

## About this Learning Path

|  |  |
| --- | --- |
| Skill level: | Advanced |
| Reading time: | 30 min |
| Last updated: | 19 Aug 2026 |

|  |  |
| --- | --- |
| Author: | Qixiang Xu , Arm |
| Arm IP: | [Cortex-A](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors) |
| Tags: | [ML](https://learn.arm.com/tag/ml) [Linux](https://learn.arm.com/tag/linux) [Python](https://learn.arm.com/tag/python) [ExecuTorch](https://learn.arm.com/tag/executorch) [XNNPACK](https://learn.arm.com/tag/xnnpack) [KleidiAI](https://learn.arm.com/tag/kleidiai) |

### Who is this for?
This is an advanced topic for developers, performance engineers, and ML framework contributors who want to benchmark and optimize KleidiAI micro-kernels within ExecuTorch to accelerate model inference on Arm64 platforms supporting SME/SME2 instructions.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Cross-compile ExecuTorch for Arm64 with XNNPACK and KleidiAI enabled, including SME/SME2 instructions
- Build and export ExecuTorch models that can be accelerated by KleidiAI using SME/SME2 instructions
- Use the executor_runner tool to run kernel workloads and collect ETDump profiling data.
- Inspect and analyze ETRecord and ETDump files using the ExecuTorch Inspector API to understand kernel-level performance behavior.

### Prerequisites
Before starting, you'll need the following prerequisites:
- An x86_64 Linux host machine running Ubuntu, with at least 15 GB of free disk space
- An Arm64 target system with support for SME or SME2 - see the Learning Path [Devices with native SME2 support](https://learn.arm.com/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started/#devices)

### Summary
You’ll cross-compile ExecuTorch with XNNPACK and KleidiAI for an `aarch64` SME or SME2 system. First, you’ll create quantized fully connected and Conv2d benchmark models that can use KleidiAI, then run them with `executor_runner`. You’ll inspect ETRecord and ETDump traces with the ExecuTorch Inspector API to validate kernel selection and compare behavior across variants.

### Frequently asked questions

**Do I need to keep my Python virtual environment active while building and exporting models?**

Yes. Keep your virtual environment active so build and runtime dependencies install and import from the same isolated location.

**What should I check on the Arm64 target before I run benchmarks?**

Verify the device supports SME or SME2. Also confirm that you deploy the AArch64 ExecuTorch binaries and libraries produced by cross-compilation.

**Which Conv2d variants does the guide benchmark with KleidiAI?**

The guide benchmarks an INT8-quantized Conv2d variant using `pqs8_qc8w_gemm`. It also benchmarks a FP32 pointwise (1×1) Conv2d variant using `pf32_gemm`.

**Which profiling files do I use with the ExecuTorch Inspector API?**

Model export creates an `ETRecord` file alongside the `.pte` model. `executor_runner` writes an `ETDump` file. Use the matching files with the ExecuTorch Inspector API to analyze kernel behavior.

**Which GEMM benchmark variants does the guide export?**

The guide exports FP16, FP32, INT8, and INT4 linear-model variants. It also exports FP16 and FP32 matrix-multiply models for comparison.