Explore options for tuning and extending AI Plank Tutor

Possible improvements to the app

You now have a complete on-device AI Plank Tutor. It detects pose landmarks from the camera, compares the learner against an instructor reference, prompts a local LLM, and speaks the result with Android text-to-speech.

You’ll explore the following areas of improvement:

• Which app parameters are worth tuning
• How model, prompt, and feedback choices affect latency
• Why fine-tuning can improve coaching quality
• What a high-level fine-tuning workflow looks like
• How the full yoga tutor app expands this simple pipeline

Tune the feedback loop

The app is intentionally small, so you can make improvements to the constants and prompt text that you’ve already used.

In MainViewModel.kt, tune how often pose data is scored:

    

        
        
private const val USER_POSE_SCORING_INTERVAL = 500L

    

A shorter interval makes the UI feel more responsive, but it can also produce more LLM prompts. A longer interval reduces work and can make coaching feel calmer.

In PoseScoreHelper.kt, tune how much pose difference is included in the prompt:

    

        
        
private const val JOINT_DIFF_FILTER = 8.0
private const val MAX_ENTRIES = 5

    

Increasing JOINT_DIFF_FILTER sends only larger joint-angle differences to the LLM. Reducing MAX_ENTRIES makes the prompt shorter and usually reduces inference time, but gives the model less context.

You can also adjust the joint list and weights in PoseScoreHelper.kt and PlankPoseData.kt. For example, a plank tutor might weight hips, shoulders, and neck alignment more strongly than wrists.

Tune the LLM response

In LlmViewModel.kt, tune how much text the model can generate:

    

        
        
private const val LLM_GENERATION_LIMIT: Int = 128

    

For this app, shorter is usually better. The tutor needs only one correction at a time, so a smaller limit can reduce latency and avoid long coaching messages.

The system prompt is the other important control:

    

        
        
private const val TUTOR_SYSTEM_PROMPT = "You are a friendly..."

    

You can use the system prompt to make the output more direct, more encouraging, or more safety-focused. Keep the prompt specific. This app works best when the model is told to give one short correction and avoid exposing the numeric joint-angle differences.

The model choice also affects the experience. Smaller models usually load and respond faster, while larger models can give better language quality. Quantization affects both speed and quality. This app uses a Q4_0 GGUF model because it’s a practical balance for mobile inference, and gets good speed-up from KleidiAI kernels.

Improve feedback quality with fine-tuning

Prompt engineering can make a general instruction model behave reasonably well, but fitness coaching has a narrow style requirement. The ideal response is short, natural, safe, and physically meaningful. Fine-tuning a small model on examples from this domain can make the output much more consistent.

A useful fine-tuning dataset should contain pairs similar to:

    

        
        
{
  "input": "I'm doing Plank pose. My joints have the following significant angle differences to the teacher's: Left hip flexion: -18, Right shoulder: 12, Neck: 10.",
  "output": "Lift your hips slightly."
}

    

The output should be the kind of phrase you want the app to speak. Avoid numbers, long explanations, and multiple corrections in one answer.

The following is a high-level workflow:

1. Define the input format.

   Keep it close to the prompt generated by PoseScoreHelper.generateLlmPromptFrom(). Include the pose name and the most important joint-angle differences.

2. Define the output style.

   Write rules for the coaching phrase: one short cue, no numeric angles, no diagnosis, no unsafe claims, and no correction for joints that aren’t in the input.

3. Generate a seed dataset.

   Start with examples you write manually. Cover common plank problems such as sagging hips, lifted hips, bent knees, rounded shoulders, bent elbows, dropped head, and uneven weight.

4. Expand the dataset with a script.

   Use a Python script to generate many input variations and call an LLM API to draft matching coaching phrases. For example, the script can choose one to three joint differences, vary the angle sizes, mirror left and right sides, and combine related issues such as hip sag with shoulder position. Ask for a structured JSON response containing one safe spoken correction. Structured outputs and batch-style requests are useful when creating larger generated datasets.

5. Review and clean the generated data.

   This is the most important step. Remove responses that mention numbers, give multiple corrections, sound unnatural when spoken, or make unsafe assumptions. Generated fitness cues should be reviewed by someone who understands the movement being coached. The dataset will often be too big to review, so generate a small dataset first to determine quality. Examine a significant section of it and determine if it needs re-generating with adjusted rules.

6. Split the dataset.

   Keep separate training, validation, and test sets. The test set should include examples from real app prompts, not only generated prompts.

7. Fine-tune the model.

   Use supervised fine-tuning or a Low-Rank adapatation (LoRA) or Quantized LoRA (QLoRA) workflow on a model that can later run on the device. Train on prompt-response examples that match the app’s runtime format. There are a number of good frameworks to work with, so you don’t need to set this up from scratch.

8. Convert and quantize for Android.

   Convert the fine-tuned model to GGUF, then quantize it to a mobile-friendly format such as Q4_0. Test quality again after quantization because quantization can change wording and consistency.

9. Evaluate on device.

   Run the app with real camera input. Check whether the model gives short, useful corrections, whether latency is acceptable, and whether repeated prompts produce varied but consistent feedback.

Fine-tuning isn’t a formal step in this Learning Path because it adds dataset design, training infrastructure, model conversion, quantization, and evaluation. However, it can make a significant quality difference to the app by making responses more accurate, natural, and stylistically consistent with the intended yoga instructor persona. Treat it as a follow-up project after the app pipeline is working.

Extend beyond one pose

The full AI Yoga Tutor app uses the same broad pipeline, but adds more product features around it.

To support multiple poses, replace the single PlankPoseData.kt object with a pose data repository that can load data from files. Each pose needs a name, instructor reference landmarks, joint weights, and display media. The prompt should include the current pose name so the LLM can choose language that fits the pose.

To show richer instructor guidance, replace plank.jpg with pose-specific images or videos. If you add videos, you’ll also need lifecycle handling for playback and a way to keep the instructor media synchronized with the current pose.

To support transitions between poses, add transition states to the app. Transitions need different feedback rules from static holds: the tutor might give timing, breathing, or movement cues rather than strict alignment corrections.

To make the tutor feel more natural, add different message types. For example:

• Correction when the score is low.
• Praise when the score is high.
• Encouragement or generic yoga advice when correction or praise might feel repetitive.
• A short rest or breathing cue between corrections.

To make the app more robust, handle interruptions and edge cases. For example, decide what should happen when the learner leaves the camera view, when no pose is detected, when the LLM is still responding, or when TTS is still speaking.

Android TTS is a good starting point for this Learning Path. To improve voice quality, you can replace Android’s built-in TextToSpeech with a higher-quality neural TTS engine. Note that this replacement can add model size, latency and runtime complexity.

Summary

You’ve now learned practical ways to tune the experience and extend it beyond the single-pose demo.

You can adapt the same structure to other fitness, movement, accessibility, or training apps. A slightly different computer vision input pipeline will have many other use cases. The key design choice is to keep each model doing the job it is good at: pose estimation extracts structured body data, scoring converts it into compact facts, and the LLM turns those facts into human-friendly coaching.