Accelerate Generative AI workloads using KleidiAI
Introduction
KleidiAI and matrix multiplication
KleidiAI in a real software stack
Quantizing and packing micro-kernels
Next Steps
Accelerate Generative AI workloads using KleidiAI
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Your framework uses KleidiAI automatically when it integrates KleidiAI and your hardware supports the required Arm instructions, such as i8mm. If either condition is missing, run the standalone example to observe the micro-kernel behavior directly.
Open
/kai/ukernels/matmul in the KleidiAI GitLab repository. The pack directory contains quantization and packing routines. Adjacent directories contain the matrix multiplication routines.Open the KleidiAI example that demonstrates the i8mm matrix multiplication micro-kernel with its packing and quantization routines. The steps identify the example before you run it.
The example exercises the i8mm matmul micro-kernel with its packing and quantization steps. It demonstrates the micro-kernel data path and performance, not framework integration.
Verify that your hardware implements the required Arm instructions, such as i8mm, and that your software selects the optimized kernels. If you use a framework, confirm that it integrates KleidiAI. Otherwise, run the standalone example to validate the micro-kernel path.