# Measure LLM inference performance with KleidiAI and SME2 on Android

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/)
- [Understand how SME2 and KleidiAI accelerate LLM inference in llama.cpp](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/introduction/)
- [Trace how KleidiAI and SME2 accelerate llama.cpp from model load to token decode](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/kleidiai_integration/)
- [Build llama.cpp with KleidiAI and SME2 enabled](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/build_llama_cpp/)
- [Measure SME2 acceleration in llama.cpp on Android](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/run_llm/)
- [Next Steps](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/performance_llama_cpp_sme2/_next-steps/)

## About this Learning Path

|  |  |
| --- | --- |
| Skill level: | Advanced |
| Reading time: | 40 min |
| Last updated: | 19 Aug 2026 |

|  |  |
| --- | --- |
| Author: | Zenon Zhilong Xiu , Arm<br>[LinkedIn](https://linkedin.com/in/zenon-zhilong-xiu-491bb398) |
| Arm IP: | [Arm C1](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors) |
| Tags: | [ML](https://learn.arm.com/tag/ml) [Android](https://learn.arm.com/tag/android) [Linux](https://learn.arm.com/tag/linux) [SME2](https://learn.arm.com/tag/sme2) [CPP](https://learn.arm.com/tag/cpp) [llama.cpp](https://learn.arm.com/tag/llama.cpp) |

### Who is this for?

This is an advanced topic for software developers, performance engineers, and AI practitioners.

### What will you learn?

After completing this Learning Path, you'll be able to perform the following tasks:

- Build llama.cpp with KleidiAI and SME2 support
- Profile large language model (LLM) inference performance on Android
- Understand how KleidiAI and SME2 accelerate LLM operators

### Prerequisites

Before starting, you'll need the following prerequisites:

- Knowledge of KleidiAI and SME2
- A Linux host machine (x86_64 or aarch64) for building `llama.cpp` with the Arm GNU Toolchain
- Git, CMake, and Android Debug Bridge (`adb`) installed on your host machine
- An Android device with Arm SME2 support for running and profiling the executable

### Summary

You’ll build `llama.cpp` with KleidiAI and SME2, then measure LLM inference on Android. First, you’ll cross-compile `llama-cli` for `aarch64`, deploy it with `adb`, and run the Llama-3.2-3B-Instruct-Q4_0 model with SME2 enabled and disabled. Then, you’ll trace acceleration through `ggml-cpu` into KleidiAI, and verify that SME2 microkernels are active before comparing the reported performance.

### Frequently asked questions

**Which compiler and toolchain should I use to build with SME2?**

Use the `aarch64` GCC cross-compile toolchain with the `aarch64-none-linux-gnu-` prefix from the Linux-hosted Arm GNU Toolchain. You need GCC version 14.2 or later to enable SME2.

**Why do I need a Linux host to build llama.cpp?**

The build uses the Linux-hosted Arm GNU Toolchain. If you work on macOS or Windows, run the commands in a Linux virtual machine, container, or development machine.

**What files must be on the Android device before measuring performance?**

Place the built `llama-cli` executable and the `Llama-3.2-3B-Instruct-Q4_0.gguf` model on the device.

**How do I verify that the SME2 path is used during inference?**

Follow the verification step that confirms SME2 microkernels are active. The `ggml-cpu` backend selects KleidiAI SME2 microkernels when the hardware supports them.

**How should I compare runs with SME2 enabled and disabled?**

Run the model on the Android device twice — once with SME2 enabled and once with it disabled. Compare the performance reported by `llama.cpp` from both runs.