Build a customer support chatbot on Android with Llama and ExecuTorch
Introduction
Create a development environment
Set up ExecuTorch
Understand Llama models
Prepare Llama models for ExecuTorch
Run the chatbot on Android
Build and run the Android chat app
Next Steps
Build a customer support chatbot on Android with Llama and ExecuTorch
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You should use the Llama 3.2 1B Instruct model. If you want to use a different model, you can adapt the same steps to other variants with minimal modification.
You should produce a
.pte file. This format is optimized for on-device inference and is loadable by the ExecuTorch runtime.Verify that
ANDROID_NDK is set to the correct path and that build/cmake/android.toolchain.cmake exists there. Also confirm your ExecuTorch dependencies are installed in the Python environment.Build ExecuTorch and its libraries for Android with KleidiAI kernels enabled through XNNPACK. Use the provided CMake configuration to include these kernels.
The Llama runner or chat app should load the exported model and generate responses to prompts on the device.