Run Vision LLM inference on Android with KleidiAI and MNN
Introduction
Background
Environment setup and prepare model
Benchmark the Vision Transformer performance with KleidiAI
Build the MNN Command-line ViT Demo
Next Steps
Run Vision LLM inference on Android with KleidiAI and MNN
Who is this for?
This Learning Path is for developers who want to run Vision Transformers (ViT) efficiently on Android.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Download a Vision Large Language Model (LLM) from Hugging Face.
- Convert the model to the Mobile Neural Network (MNN) framework.
- Install an Android demo application using the model to run an inference.
- Compare inference performance with and without KleidiAI Arm-optimized micro-kernels.
Prerequisites
Before starting, you will need the following:
- A development machine with Android Studio installed.
- A smartphone running Android with support for
i8mmanddotprodinstructions.