Who is this for?
This is an advanced topic for software developers, performance engineers, and AI practitioners.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Explain how a KleidiAI microkernel performs matrix multiplication (matmul) with quantized data
- Identify how SME2 INT8 MOPA (matrix outer product accumulate) instructions map to matmul work
- Trace how quantization and packing feed an SME2 matmul microkernel (using GGML Q4_0 and llama.cpp call stacks as a concrete example)
- Perform basic hands-on checks (source inspection and optional disassembly) to confirm where SME2 instructions appear
Prerequisites
Before starting, you will need the following:
- Basic understanding of general matrix multiplication (GEMM) and matmul operations
- Basic understanding of quantization concepts for neural networks
- (Optional) Access to an Arm CPU with SME2 support (Linux or Android) for hands-on verification steps
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You’ll examine how a KleidiAI SME2 INT8 MOPA microkernel performs quantized matrix multiplication on Arm CPUs. First, you’ll learn how tiling and packing parameters control the kernel, decode its name, and interpret its work unit. Then, you’ll follow a
GGML Q4_0 example and inspect source or disassembly to identify the SME2 instructions.Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
No. SME2 hardware is only required for the optional hands-on verification steps.
Inspect the microkernel’s inner loop for SME2 outer product accumulate instructions that update the ZA storage. Optional disassembly can also confirm where these instructions appear.
These tiling and block parameters define the output tile shape and how many K elements are processed per step. Pack A and B so their layouts align with these increments and the kernel’s access pattern.
It indicates that one inner-loop iteration computes an intermediate 1VL by 4VL submatrix of the output. The actual element counts depend on the device’s SME2 streaming vector length.
The RHS buffer layout changes to the format expected by the SME2 microkernel so the inner loop can stream data efficiently. The kernel then consumes the packed weights without additional rearrangement.