Build an on-device AI fitness tutor app on Android
Introduction
Set up AI Plank Tutor
Get pose landmarks from camera
Add scoring logic for the plank pose
Turn pose data into a prompt for the local LLM
Run the LLM locally with AI Chat
Connect coaching corrections to Android text-to-speech
Explore options for tuning and extending AI Plank Tutor
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  1 hr 30 min
Last updated:  22 Jul 2026
Skill level:
  Introductory
Reading time:
  1 hr 30 min
Last updated:
  22 Jul 2026
Author:Ben Clark, Arm
Arm IP:Cortex-A Arm AI Chat library
Tags: ML Android Android Studio Kotlin CameraX MediaPipe LLM Neon SVE2 SME2
Author:
  • Ben Clark, Arm
Arm IP:
Cortex-A Arm AI Chat library
Tags:
ML Android Android Studio Kotlin CameraX MediaPipe LLM Neon SVE2 SME2

Who is this for?

This Learning Path is for Android developers who want to explore creating an ML or Generative AI pipeline, including camera input, local Large Language Model (LLM) inference, and speech.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Detect human pose landmarks from live Android camera frames with MediaPipe.
  • Structure data and build compact prompts that turn raw data into useful LLM input.
  • Run a mobile-sized LLM on-device with Arm's AI Chat library.
  • Generate output as speech with Text-To-Speech.

Prerequisites

Before starting, you will need the following:

  • A development machine with Android Studio installed.
  • A recent Arm-powered Android phone in Developer Mode, with USB debugging enabled, a USB data cable, and at least 5 GB of free storage for the GGUF model import.
  • Android Debug Bridge (adb), included with the Android SDK platform tools.
  • Basic familiarity with Kotlin and Android app development.

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build AI Plank Tutor, an on-device Android app that evaluates a static plank pose and delivers spoken correction. You’ll configure CameraX and MediaPipe Pose Landmarker to extract landmarks from live camera frames, calculate joint angles, and compute a weighted score against an instructor reference. Then, you’ll turn the largest joint-angle differences into a compact prompt for a local LLM. Finally, you’ll integrate Arm AI Chat for GGUF inference and connect Android text-to-speech so the app speaks each correction. By the end, you’ll have a complete on-device pipeline from camera input to spoken feedback.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Where should I add the CameraX initialization code?
Open ui/MainActivity.kt and replace the TODO in setUpCamera() with the provided ProcessCameraProvider snippet. When the provider is ready, the callback stores it and calls bindCameraUseCases().
What do I do if the camera doesn't detect pose landmarks?
After CameraX initializes and binds its use cases, MediaPipe Pose Landmarker should produce landmarks for each frame. If the camera preview appears but no pose landmarks are detected, make sure that you granted camera permission and that your full body is visible in the frame. Then, check Logcat for PoseLandmarkerHelper errors. If the preview doesn’t appear, confirm that bindCameraUseCases() runs after ProcessCameraProvider becomes available.
What defines the reference pose and weighting used to score the plank?
data/PlankPoseData.kt contains referenceLandmarks for the instructor pose and angleWeights that set the importance of each joint. The app uses these values to calculate joint angles and a weighted score.
How do I control which joint differences are sent to the LLM?
ui/landmarker/PoseScoreHelper.kt maps angles to names with KEY_ANGLE_NAMES and formats the largest differences in angleDifference(). Use the filter and maxEntries parameters to set a threshold and limit the entries included in the prompt.
Where do I add the AI Chat dependency and which model format should I use?
Add implementation("com.arm:ai-chat:0.1.0") to the dependencies block in app/build.gradle, then sync the project.
Next