Run Vision LLM inference on Android with KleidiAI and MNN
Introduction
Background
Environment setup and prepare model
Benchmark the Vision Transformer performance with KleidiAI
Build the MNN Command-line ViT Demo
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  30 min
Last updated:  21 Aug 2026
Skill level:
  Introductory
Reading time:
  30 min
Last updated:
  21 Aug 2026
Authors:Shuheng Deng, Arm
Yiyang Fan, Arm
Arm IP:Cortex-A
Tags: ML Android Android Studio KleidiAI
Authors:
  • Shuheng Deng, Arm
  • Yiyang Fan, Arm
Arm IP:
Cortex-A
Tags:
ML Android Android Studio KleidiAI

Who is this for?

This Learning Path is for developers who want to run Vision Transformers (ViT) efficiently on Android.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Download a Vision Large Language Model (LLM) from Hugging Face.
  • Convert the model to the Mobile Neural Network (MNN) framework.
  • Install an Android demo application using the model to run an inference.
  • Compare inference performance with and without KleidiAI Arm-optimized micro-kernels.

Prerequisites

Before starting, you will need the following:

  • A development machine with Android Studio installed.
  • A smartphone running Android with support for i8mm and dotprod instructions.

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll run the Qwen2.5-VL-3B-Instruct-MNN vision model on an Android device with MNN and KleidiAI. First, you’ll install the Android tools, download a pre-quantized MNN model, and build the Android Studio demo. Then, you’ll prepare an image, build and run the MNN command-line demo, enable KleidiAI, rebuild the binaries, and compare the reported benchmark timings.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Which NDK and CMake do I need, and how do I install them?
To match the tested setup, use Android NDK 28.0.12916984 and CMake 4.0.0-rc1. In Android Studio, select Tools > SDK Manager, open SDK Tools, then select NDK (Side by side) and CMake. On Ubuntu or Debian, install cmake and git-lfs with the provided command.
How do I clone and open the project in Android Studio?
Run git clone https://gitlab.arm.com/kleidi/kleidi-examples/vision-language-models. In Android Studio, select File > Open, choose the vision-language-models directory, and select Open. Android Studio then builds the project.
Where should I put the example image and what name should it have?
Rename the image to example.png, then run adb push example.png /data/local/tmp/ to copy it to your device.
How do I compare inference with and without KleidiAI micro-kernels?
Build and run the MNN command-line demo without KleidiAI first. Then add the CPU_ENABLE_KLEIDIAI runtime hint, rebuild the binaries, replace them on your device, and run the same inference command again to compare the reported timings.
When is the model converted to MNN, and how do I know it worked?
Model conversion is optional. By default, you download the pre-quantized Qwen2.5-VL-3B-Instruct-MNN model. If you convert the model with llmexport, verify that the Qwen2.5-VL-3B-Instruct-convert-4bit-64qblock directory is at least 2 GB.
Next