Add an LLM to your Android app with Arm's AI Chat library
Introduction
Create the Android project
Configure the AI Chat library dependency
Create the UI layouts and message adapter
Implement the main activity logic
Download a model and run the app
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  15 min
Last updated:  05 Aug 2026
Skill level:
  Introductory
Reading time:
  15 min
Last updated:
  05 Aug 2026
Author:Ben Clark, Arm
Arm IP:Cortex-A
Tags: ML Android Kotlin Neon SVE2 SME2 LLM
Author:
  • Ben Clark, Arm
Arm IP:
Cortex-A
Tags:
ML Android Kotlin Neon SVE2 SME2 LLM

Who is this for?

This is an introductory topic for developers who want to add a local, on-device LLM chat experience using Arm's AI Chat library, Kotlin, and Android Studio.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Create a simple Android chatbot app scaffold in Android Studio
  • Load a mobile-friendly GGUF model on-device and run streamed chat inference

Prerequisites

Before starting, you will need the following:

  • An Android development environment with Android Studio installed
  • An Android phone for testing, in Developer Mode, with USB cable for connection
  • Basic familiarity with Kotlin and Android app development

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build a minimal Android chatbot that runs a local GGUF model with Arm’s AI Chat library. First, you’ll configure the Android Studio project, add the Maven dependency, and create a chat UI. Then, you’ll connect the MainActivity class to load a mobile-friendly model and stream responses. Finally, you’ll download a GGUF model file on the device and run the app to verify on-device output.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Which Gradle file should I edit to add the AI Chat dependency?
Add the dependency in the app module’s build.gradle.kts, not the project-level build.gradle.kts. Place the implementation for com.arm:ai-chat:0.1.0 in the dependencies block.
How do I check my repository configuration so the library resolves?
Open settings.gradle.kts and confirm the top-level repositories include google() and mavenCentral(). With both present, you can resolve the AI Chat library from Maven Central.
Which layout file do I replace for the chat UI?
Replace activity_main.xml in app/src/main/res/layout with the provided XML. The XML defines a status area, a message list, and a text input with a send button.
How do I choose a GGUF model that fits my device?
Select a model that is significantly smaller than your phone’s available RAM. A mobile-friendly example is google_gemma-3-4b-it-Q4_0.gguf, a Q4_0‑quantized Gemma 3 4B model that works well with Arm’s KleidiAI and benefits devices with SME2, SVE2, or Neon capabilities.
What result should I expect when I run the app?
After you provide the GGUF file on the device, you should see the app load the model and stream chat responses into the message list. Check the status area during generation to confirm progress.
Next