Build an Android chat app with Llama, KleidiAI, ExecuTorch, and XNNPACK
Introduction
Create a development environment
ExecuTorch Setup
Understanding Llama models
Prepare Llama models for ExecuTorch
Run Benchmark on Android phone
Build and Run Android chat app
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

In this section, you will use a Android demo application to demonstrate local inference with ExecuTorch.

Build the Android Archive (AAR)

  1. Open a terminal window and navigate to the root directory of the executorch repository.

  2. If you haven’t already, set the following environment variables:

        

        
        
export ANDROID_NDK=$ANDROID_HOME/ndk/29.0.14206865/
export ANDROID_ABI=arm64-v8a
export ANDROID_SDK=$ANDROID_HOME
Note

<path_to_android_ndk> is the root for the NDK, which is usually under ~/Library/Android/sdk/ndk/XX.Y.ZZZZZ for macOS, and contains NOTICE and README.md. Make sure you can confirm <path_to_android_ndk>/build/cmake/android.toolchain.cmake is available for CMake to cross-compile.

  1. Run the following command to set up the required JNI library:

        

        
        
sh scripts/build_android_library.sh

Getting models

Make sure the exported model and tokenizer are copied to the Android phone:

Option 1: using adb

  1. Check if the files are available on the phone:
    

        
        
adb shell "ls -la /data/local/tmp/llama/"
  1. If not, copy them:
    

        
        
adb shell mkdir -p /data/local/tmp/llama
adb push <model.pte> /data/local/tmp/llama/
adb push <tokenizer.bin> /data/local/tmp/llama/

Option 2: Using Android Studio

  1. Use Android Studio’s device explorer to look for the model files.

Image Alt Text:Device ExplorerFigure 1. Android Studio Device Explorer

  1. Upload the files.

If the files are not on the device, use the device explorer to copy them.

Image Alt Text:Files UploadFigure 2. Android Studio upload files using Device Explorer

Build the Android Package Kit

Before starting, you need to obtain the example app by cloning executorch-examples:

    

        
        
git clone https://github.com/meta-pytorch/executorch-examples.git

Option 1: Using Android Studio

This is the recommended option.

  1. Open Android Studio and select Open an existing Android Studio project and navigate to open executorch-examples/llm/android/LlamaDemo.

  2. Run the app (^R). This builds and launches the app on the phone.

Option 2: Command line

Without Android Studio UI, you can run gradle directly to build the app. You need to set up the Android SDK path and invoke gradle. Navigate to the newly cloned executorch-examples repository.

    

        
        
pushd llm/android/LlamaDemo
./gradlew :app:installDebug
popd

You should now see a running app on your phone that looks like this:

Image Alt Text:First example prompt Image Alt Text:Second example prompt

Back
Next