Benchmark a KleidiAI micro-kernel in ExecuTorch
Introduction
Set up your environment
Cross-Compile ExecuTorch for the AArch64 platform
Accelerate ExecuTorch operators with KleidiAI micro-kernels
Create and quantize linear layer benchmark model
Create and quantize convolution layer benchmark model
Create matrix multiply layer benchmark model
Run model and generate the ETDump
Analyze ETRecord and ETDump
Next Steps
Benchmark a KleidiAI micro-kernel in ExecuTorch
Introduction
Set up your environment
Cross-Compile ExecuTorch for the AArch64 platform
Accelerate ExecuTorch operators with KleidiAI micro-kernels
Create and quantize linear layer benchmark model
Create and quantize convolution layer benchmark model
Create matrix multiply layer benchmark model
Run model and generate the ETDump
Analyze ETRecord and ETDump
Next Steps
Who is this for?
This is an advanced topic for developers, performance engineers, and ML framework contributors who want to benchmark and optimize KleidiAI micro-kernels within ExecuTorch to accelerate model inference on Arm64 platforms supporting SME/SME2 instructions.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Cross-compile ExecuTorch for Arm64 with XNNPACK and KleidiAI enabled, including SME/SME2 instructions
- Build and export ExecuTorch models that can be accelerated by KleidiAI using SME/SME2 instructions
- Use the executor_runner tool to run kernel workloads and collect ETDump profiling data.
- Inspect and analyze ETRecord and ETDump files using the ExecuTorch Inspector API to understand kernel-level performance behavior.
Prerequisites
Before starting, you will need the following:
- An x86_64 Linux host machine running Ubuntu, with at least 15 GB of free disk space
- An Arm64 target system with support for SME or SME2 - see the Learning Path Devices with native SME2 support
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
aarch64 SME or SME2 system. First, you’ll create quantized fully connected and Conv2d benchmark models that can use KleidiAI, then run them with executor_runner. You’ll inspect ETRecord and ETDump traces with the ExecuTorch Inspector API to validate kernel selection and compare behavior across variants.Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
pqs8_qc8w_gemm. It also benchmarks a FP32 pointwise (1×1) Conv2d variant using pf32_gemm.ETRecord file alongside the .pte model. executor_runner writes an ETDump file. Use the matching files with the ExecuTorch Inspector API to analyze kernel behavior.