# [Run LLM inference on Android with KleidiAI, MediaPipe, and XNNPACK](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/)

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/)
- [Install dependencies](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/1-x86-install-dependencies/)
- [Run the Gemma 2B model using MediaPipe with XNNPACK](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/2-run-gemma-2b/)
- [Benchmark the Gemma 2B Model with KleidiAI](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/3-benchmark-gemma-i8mm/)
- [Next Steps](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/_next-steps/)

## About this Learning Path

| Skill level: | Advanced |
|--------------|----------|
| Reading time: | 1 hr     |
| Last updated: | 19 Aug 2026 |

### Authors:
- Pareena Verma, Arm  
  [GitHub](https://github.com/pareenaverma)  
  [LinkedIn](https://linkedin.com/in/pareena-verma-7853607)
- Joe Stech, Arm  
  [GitHub](https://github.com/JoeStech)  
  [LinkedIn](https://linkedin.com/in/joestech)
- Adnan AlSinan, Arm

### Arm IP:
[Cortex-A](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors)

### Tags:
- [ML](/tag/ml)
- [Linux](/tag/linux)
- [Java](/tag/java)
- [MediaPipe](/tag/mediapipe)
- [Android SDK](/tag/android-sdk)
- [Android NDK](/tag/android-ndk)
- [Bazel](/tag/bazel)
- [XNNPACK](/tag/xnnpack)
- [Hugging Face](/tag/hugging-face)

### Who is this for?
This is an advanced topic for Android developers who want to efficiently run LLMs on-device.

### What will you learn?
Upon completion of this Learning Path, you will be able to:
- Install the prerequisites for cross-compiling new inference engines for Android.
- Run LLM inference on an Android device with the Gemma 2B model using the Google AI Edge's MediaPipe framework.
- Benchmark LLM inference speed with and without the KleidiAI-enhanced Arm i8mm processor feature.

### Prerequisites
Before starting, you will need the following:
- An x86_64 Linux machine running Ubuntu with approximately 500 MB of free space, or a docker daemon that can build and run a provided x86_64 Dockerfile.
- An Android phone with support for i8mm (tested on Google Pixel 8 Pro).

### Summary
You’ll cross-compile MediaPipe with XNNPACK for Android `arm64` and run Gemma 2B on an i8mm-capable device. First, you’ll build and verify the binary with Bazel, then run inference. After running inference, you’ll create a second build without i8mm and benchmark both variants to assess the effect of KleidiAI integration on supported Android hardware.

### Frequently asked questions
<details>
<summary>Which installation option should I choose for dependencies?</summary>
Use the Docker option if you have a working Docker daemon and prefer an isolated, reproducible environment. Choose the native Ubuntu option if you want to install dependencies directly on your `x86_64` Linux host.
</details>

<details>
<summary>How do I confirm the Bazel build created the Android CPU inference binary?</summary>
List the Bazel output directory. You should see `llm_inference_engine_cpu_main` under `bazel-bin/mediapipe/tasks/cc/genai/inference/c/`.
</details>

<details>
<summary>Which Bazel options enable the i8mm path with KleidiAI for Android arm64?</summary>
Build the target `mediapipe/tasks/cc/genai/inference/c:llm_inference_engine_cpu_main` with `--config=android_arm64` and `--define=xnn_enable_arm_i8mm=true`. These options select the Android `arm64` build and enable the i8mm path integrated through XNNPACK.
</details>

<details>
<summary>Which options build `llm_test` with i8mm but without KleidiAI?</summary>
In the `llm_test` Bazel command, use `--define=xnn_enable_arm_i8mm=true` with `--define=xnn_enable_kleidiai=false`. Retain `-c opt`, `--config=android_arm64`, `--dynamic_mode=off`, and the `mediapipe/tasks/cc/genai/inference/utils/xnn_utils:llm_test` target.
</details>

<details>
<summary>What do the two `llm_test` benchmark runs compare?</summary>
Both builds enable i8mm. The first disables KleidiAI micro-kernels, while the second enables them by default, so you can compare the effect of KleidiAI on your device.
</details>