Run ExecuTorch Llama 3.2 1B Instruct on an Android phone with Vulkan
Introduction
Prepare a Linux host and Android device for ExecuTorch Vulkan
Install ExecuTorch and download the model on the Linux host
Export the Vulkan PTE
Build the Android Vulkan runtime
Deploy and measure Llama 3.2 with ExecuTorch Vulkan on Android
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

Install the Vulkan SDK on the Linux host

The Android Vulkan build needs a host glslc to compile GLSL shaders to SPIR-V.

Install a recent LunarG Vulkan SDK on the Linux host:

    

        
        
sudo apt update
sudo apt install -y curl xz-utils
mkdir -p "$HOME/vulkan"
cd "$HOME/vulkan"

SDK_VERSION=$(curl -fsSL https://vulkan.lunarg.com/sdk/latest/linux.txt)
curl -fL -o vulkan_sdk.tar.xz \
  "https://sdk.lunarg.com/sdk/download/${SDK_VERSION}/linux/vulkan_sdk.tar.xz"
tar xf vulkan_sdk.tar.xz
source "$HOME/vulkan/$SDK_VERSION/setup-env.sh"

Validate that the host SDK is active:

    

        
        
which glslc
glslc --version
echo "$VULKAN_SDK"

Ensure that which glslc resolves to the host SDK, not to an incompatible Android NDK copy.

Configure and build ExecuTorch for Android plus Vulkan

From the ExecuTorch checkout, run the following commands:

    

        
        
cd $HOME/executorch
rm -rf cmake-out-android-so

cmake . \
  -DCMAKE_INSTALL_PREFIX=cmake-out-android-so \
  -DCMAKE_TOOLCHAIN_FILE="$ANDROID_NDK/build/cmake/android.toolchain.cmake" \
  -DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON \
  --preset android-arm64-v8a \
  -DANDROID_PLATFORM=android-28 \
  -DPYTHON_EXECUTABLE="$(which python)" \
  -DCMAKE_BUILD_TYPE=Release \
  -DEXECUTORCH_PAL_DEFAULT=posix \
  -DEXECUTORCH_BUILD_LLAMA_JNI=OFF \
  -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON \
  -DEXECUTORCH_BUILD_VULKAN=ON \
  -DEXECUTORCH_BUILD_TESTS=OFF \
  -B cmake-out-android-so

cmake --build cmake-out-android-so \
  -j4 \
  --target install \
  --config Release

Build the Android llama_main runner

Build the example runner against the installed Android runtime:

    

        
        
cmake examples/models/llama \
  -DCMAKE_INSTALL_PREFIX=cmake-out-android-so \
  -DCMAKE_TOOLCHAIN_FILE="$ANDROID_NDK/build/cmake/android.toolchain.cmake" \
  -DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON \
  -DEXECUTORCH_ENABLE_LOGGING=ON \
  -DANDROID_ABI=arm64-v8a \
  -DANDROID_PLATFORM=android-28 \
  -DCMAKE_BUILD_TYPE=Release \
  -DPYTHON_EXECUTABLE="$(which python)" \
  -B cmake-out-android-so/examples/models/llama

cmake --build cmake-out-android-so/examples/models/llama \
  -j4 \
  --config Release

Verify the output binary:

    

        
        
file cmake-out-android-so/examples/models/llama/llama_main

The output is similar to:

    

        
        ELF 64-bit LSB pie executable, ARM aarch64, ... interpreter /system/bin/linker64

What you’ve accomplished and what’s next

You’ve now built the Android runtime and llama_main runner.

Next, you’ll deploy the build artifacts on the Android device.

Back
Next