# Run LLM inference on Android with KleidiAI, MediaPipe, and XNNPACK

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/)
- [Install dependencies](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/1-x86-install-dependencies/)
- [Run the Gemma 2B model using MediaPipe with XNNPACK](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/2-run-gemma-2b/)
- [Benchmark the Gemma 2B Model with KleidiAI](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/3-benchmark-gemma-i8mm/)
- [Next Steps](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/_next-steps/)

## About this Learning Path

|  |  |
| --- | --- |
| Skill level: | Advanced |
| Reading time: | 1 hr |
| Last updated: | 19 Aug 2026 |

| Authors:         |                       |
|------------------|-----------------------|
|                  | Pareena Verma, Arm    |
|                  | Joe Stech, Arm        |
|                  | Adnan AlSinan, Arm    |

|  |  |
| --- | --- |
| Authors: | Pareena Verma , Arm<br>[GitHub](https://github.com/pareenaverma)<br>[LinkedIn](https://linkedin.com/in/pareena-verma-7853607)<br>Joe Stech , Arm<br>[GitHub](https://github.com/JoeStech)<br>[LinkedIn](https://linkedin.com/in/joestech)<br>Adnan AlSinan , Arm |
| Arm IP: | [Cortex-A](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors) |
| Tags: | [ML](https://learn.arm.com/tag/ml) [Linux](https://learn.arm.com/tag/linux) [Java](https://learn.arm.com/tag/java) [MediaPipe](https://learn.arm.com/tag/mediapipe) [Android SDK](https://learn.arm.com/tag/android-sdk) [Android NDK](https://learn.arm.com/tag/android-ndk) [Bazel](https://learn.arm.com/tag/bazel) [XNNPACK](https://learn.arm.com/tag/xnnpack) [Hugging Face](https://learn.arm.com/tag/hugging-face) |

### Who is this for?
This is an advanced topic for Android developers who want to efficiently run LLMs on-device.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Install the prerequisites for cross-compiling new inference engines for Android.
- Run LLM inference on an Android device with the Gemma 2B model using the Google AI Edge's MediaPipe framework.
- Benchmark LLM inference speed with and without the KleidiAI-enhanced Arm i8mm processor feature.

### Prerequisites
Before starting, you'll need the following prerequisites:
- An x86_64 Linux machine running Ubuntu with approximately 500 MB of free space, or a docker daemon that can build and run a provided x86_64 Dockerfile.
- An Android phone with support for i8mm (tested on Google Pixel 8 Pro).

### Summary
You’ll cross-compile MediaPipe with XNNPACK for Android `arm64` and run Gemma 2B on an i8mm-capable device. First, you’ll build and verify the binary with Bazel, then run inference. After running inference, you’ll create a second build without i8mm and benchmark both variants to assess the effect of KleidiAI integration on supported Android hardware.

### Frequently asked questions

**Which installation option should I choose for dependencies?**

Use the Docker option if you have a working Docker daemon and prefer an isolated, reproducible environment. Choose the native Ubuntu option if you want to install dependencies directly on your `x86_64` Linux host.

**How do I confirm the Bazel build created the Android CPU inference binary?**

List the Bazel output directory. You should see `llm_inference_engine_cpu_main` under `bazel-bin/mediapipe/tasks/cc/genai/inference/c/`.

**Which Bazel options enable the i8mm path with KleidiAI for Android arm64?**

Build the target `mediapipe/tasks/cc/genai/inference/c:llm_inference_engine_cpu_main` with `--config=android_arm64` and `--define=xnn_enable_arm_i8mm=true`. These options select the Android `arm64` build and enable the i8mm path integrated through XNNPACK.

<summary>Which options build `llm_test` with i8mm but without KleidiAI?</summary>
In the `llm_test` Bazel command, use `--define=xnn_enable_arm_i8mm=true` with `--define=xnn_enable_kleidiai=false`. Retain `-c opt`, `--config=android_arm64`, `--dynamic_mode=off`, and the `mediapipe/tasks/cc/genai/inference/utils/xnn_utils:llm_test` target.

<summary>What do the two `llm_test` benchmark runs compare?</summary>
Both builds enable i8mm. The first disables KleidiAI micro-kernels, while the second enables them by default, so you can compare the effect of KleidiAI on your device.