Build a customer support chatbot on Android with Llama and ExecuTorch
Introduction
Create a development environment
Set up ExecuTorch
Understand Llama models
Prepare Llama models for ExecuTorch
Run the chatbot on Android
Build and run the Android chat app
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  1 hr
Last updated:  05 Aug 2026
Skill level:
  Introductory
Reading time:
  1 hr
Last updated:
  05 Aug 2026
Author:Parichay Das
Arm IP:Cortex-A
Tags: ML macOS Linux Android Java Python ExecuTorch
Author:
  • Parichay Das
Arm IP:
Cortex-A
Tags:
ML macOS Linux Android Java Python ExecuTorch

Who is this for?

This is an introductory topic for software developers interested in building an on-device customer support chatbot for Android using Meta's Llama models and the ExecuTorch runtime.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Set up a development environment for building and deploying ExecuTorch-based apps on Android
  • Describe how ExecuTorch uses KleidiAI kernels to accelerate performance on Arm-based platforms
  • Export a Llama 3.2 model to .pte format optimized for on-device inference
  • Run a Llama model on an Arm-powered Android phone and verify inference performance
  • Build and run an Android chat app configured as a customer support assistant

Prerequisites

Before starting, you will need the following:

  • An Apple M1/M2/M3 development machine, or a Linux machine with at least 16GB of RAM
  • An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM
  • A USB cable to connect your smartphone to your development machine
  • Android Debug Bridge (adb) installed. Follow the steps in adb to install Android SDK Platform Tools
  • Java 17 JDK. Follow the steps in Java SE 17 Archive Downloads to download and install JDK for your host
  • Python 3.10 or later
  • A Hugging Face account with access to Meta Llama models

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build and deploy an on-device Android customer support chatbot with Meta’s Llama 3.2, ExecuTorch, and KleidiAI. First, you’ll set up the environment and choose a model and quantization approach. Then, you’ll obtain Llama 3.2 1B Instruct weights and export them to .pte. Finally, you’ll cross-compile the runner with the Android NDK and enable KleidiAI through XNNPACK, then deploy to an Arm phone and run inference.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Which Llama model variant should I download, and can I use a different one?
You should use the Llama 3.2 1B Instruct model. If you want to use a different model, you can adapt the same steps to other variants with minimal modification.
What output should I have after exporting the model for ExecuTorch?
You should produce a .pte file. This format is optimized for on-device inference and is loadable by the ExecuTorch runtime.
What should I check if the Android cross-compile step fails to find the toolchain?
Verify that ANDROID_NDK is set to the correct path and that build/cmake/android.toolchain.cmake exists there. Also confirm your ExecuTorch dependencies are installed in the Python environment.
How do I make sure KleidiAI acceleration is included in my Android build?
Build ExecuTorch and its libraries for Android with KleidiAI kernels enabled through XNNPACK. Use the provided CMake configuration to include these kernels.
What result should I expect when I run the chatbot on the phone?
The Llama runner or chat app should load the exported model and generate responses to prompts on the device.
Next