Run Vision LLM inference on Android with KleidiAI and MNN
Introduction
Background
Environment setup and prepare model
Benchmark the Vision Transformer performance with KleidiAI
Build the MNN Command-line ViT Demo
Next Steps
Run Vision LLM inference on Android with KleidiAI and MNN
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
To match the tested setup, use Android NDK
28.0.12916984 and CMake 4.0.0-rc1. In Android Studio, select Tools > SDK Manager, open SDK Tools, then select NDK (Side by side) and CMake. On Ubuntu or Debian, install cmake and git-lfs with the provided command.Run
git clone https://gitlab.arm.com/kleidi/kleidi-examples/vision-language-models. In Android Studio, select File > Open, choose the vision-language-models directory, and select Open. Android Studio then builds the project.Rename the image to
example.png, then run adb push example.png /data/local/tmp/ to copy it to your device.Build and run the MNN command-line demo without KleidiAI first. Then add the
CPU_ENABLE_KLEIDIAI runtime hint, rebuild the binaries, replace them on your device, and run the same inference command again to compare the reported timings.Model conversion is optional. By default, you download the pre-quantized
Qwen2.5-VL-3B-Instruct-MNN model. If you convert the model with llmexport, verify that the Qwen2.5-VL-3B-Instruct-convert-4bit-64qblock directory is at least 2 GB.