Build an Android chat application with ONNX Runtime API
Introduction
Create a development environment
Build ONNX Runtime
Build ONNX Runtime Generate() API
Run a benchmark on an Android phone
Build and run an Android chat app
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr
Last updated:  05 Aug 2026
Skill level:
  Advanced
Reading time:
  1 hr
Last updated:
  05 Aug 2026
Author:Koki Mitsunami, Arm
Arm IP:Cortex-A
Tags: ML Windows Android Kotlin CPP ONNX Runtime Android Hugging Face
Author:
  • Koki Mitsunami, Arm
Arm IP:
Cortex-A
Tags:
ML Windows Android Kotlin CPP ONNX Runtime Android Hugging Face

Who is this for?

This is an advanced topic for software developers interested in learning how to build an Android chat app with ONNX Runtime and ONNX Runtime Generate() API.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Build ONNX Runtime and ONNX Runtime generate() API for Android.
  • Run a Phi-3 model using ONNX Runtime on an Arm-based smartphone.

Prerequisites

Before starting, you will need the following:

  • A Windows x86_64 development machine with at least 16GB of RAM.
  • An Android phone with at least 8GB of RAM. This learning path was tested on Samsung Galaxy S24.

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build ONNX Runtime and its generate() API for Android on a Windows host, then deploy a Phi-3 model to an Arm-based smartphone. First, you’ll configure the Android NDK, cross-compile the runtime and text-generation components, and build a C model runner with CMake and Ninja. Then, you’ll launch the Android demo app in Android Studio to validate on-device inference.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I point CMake to the correct Android NDK for these builds?
Use the CMAKE_TOOLCHAIN_FILE option with the android.toolchain.cmake path from the installed NDK. Ensure the path and version match the NDK you installed.
Which header files do I need to copy before configuring the C model runner?
Copy src\ort_genai.h and src\ort_genai_c.h into examples\c\include. If these headers are missing, the model runner configuration will fail with include errors.
How do I know ONNX Runtime and the generate() API built successfully?
The build is successful when the configure and build stages complete without errors and produce Android-targeted binaries and headers. You should then be able to configure and build the examples\c model runner without missing include or link errors.
What result should I expect when I run the Phi-3-mini benchmark on my phone?
The run executes on the Android device and prints performance metrics. Proceed when the metrics appear and the process exits cleanly without runtime errors.
Which commit of the demo app repository should I check out in Android Studio?
Check out commit 7a635daae48450ff142e5c0848a564b245f04112. You might be able to use a later commit, but the Learning Path was tested with that specific commit.
Next