Vision LLM inference on Android with KleidiAI and MNN

About this Learning Path

Who is this for?

This Learning Path is for developers who want to run Vision Transformers (ViT) efficiently on Android.

What will you learn?

Upon completion of this Learning Path, you will be able to:

• Download a Vision Large Language Model (LLM) from Hugging Face.
• Convert the model to the Mobile Neural Network (MNN) framework.
• Install an Android demo application using the model to run an inference.
• Compare inference performance with and without KleidiAI Arm-optimized micro-kernels.

Prerequisites

Before starting, you will need the following:

• A development machine with Android Studio installed.
• A smartphone running Android with support for i8mm and dotprod instructions.