# [Turn pose data into a prompt for the local LLM](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/4-data-to-prompt/)

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/)
- [Set up AI Plank Tutor](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/1-introduce-plank-tutor/)
- [Get pose landmarks from camera](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/2-camera-pose-landmarks/)
- [Add scoring logic for the plank pose](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/3-plank-score/)
- [Turn pose data into a prompt for the local LLM](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/4-data-to-prompt/)
- [Run the LLM locally with AI Chat](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/5-llm-ai-chat/)
- [Connect coaching corrections to Android text-to-speech](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/6-android-tts/)
- [Explore options for tuning and extending AI Plank Tutor](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/7-extend/)
- [Next Steps](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/_next-steps/)

## Format the largest angle differences

Open `ui/landmarker/PoseScoreHelper.kt`.

The file already contains `KEY_ANGLE_NAMES`, which maps each calculated angle to a human-readable joint name. The order of `KEY_ANGLE_NAMES` matches the order of the angles returned by `extractAnglesFrom()`.

Replace the TODO in `angleDifference()` with the following code:

```kotlin
fun angleDifference(
    correctAngles: List<Double>,
    userAngles: List<Double>,
    filter: Double = -1.0,
    maxEntries: Int = Int.MAX_VALUE
): Map<String, String> {
    require(correctAngles.isNotEmpty()) { "Correct angles cannot be empty." }
    require(correctAngles.size == userAngles.size) { "Correct and user angle counts must match." }

    return correctAngles.zip(userAngles)
        .zip(KEY_ANGLE_NAMES) { correct, user -> 
            val difference = correct - user
            Triple(name, difference, "%.0f".format(difference))
        }
        .filter { (_, difference, _) -> filter <= 0 || abs(difference) >= filter }
        .sortedByDescending { abs(it.second) }
        .take(maxEntries)
        .associate { (name, _, formatted) -> name to formatted }
}
```

The function pairs each reference angle with the corresponding learner angle, calculates the difference, and sorts by the absolute size of that difference.

The returned map uses the joint name as the key and a rounded string angle difference as the value. The string value is useful because the next function can insert it directly into the prompt.

> **Note**  
> The sign of the difference is important. `difference = reference - learner`, so a positive value means the learner’s angle is smaller than the reference and needs to straighten. A negative value means the learner’s angle is larger than the reference and needs to bend more.

## Generate the LLM prompt

The LLM won’t receive images or raw landmarks. It’ll receive a short list of the largest pose differences, such as:

```plaintext
I'm doing Plank pose. My joints have the following significant angle differences to the teacher's: Left hip flexion: -18, Right shoulder: 12
```

This keeps the prompt small and gives the model only the facts it needs to produce one coaching cue.

This is the main design pattern in the app: the vision model turns camera frames into structured pose data, deterministic Kotlin code turns that data into a small set of facts, and the LLM takes these facts and returns appropriate advice.

Replace the TODO in `generateLlmPromptFrom()` with the following code:

```kotlin
fun generateLlmPromptFrom(
    referenceAngles: List<Double>,
    userAngles: List<Double>,
    poseName: String
): String {
    val filteredAngles = angleDifference(
        referenceAngles,
        userAngles,
        JOINT_DIFF_FILTER,
        MAX_ENTRIES
    )

    if (filteredAngles.isEmpty()) {
        return "I'm doing $poseName pose. My joint angles are close to the teacher's reference; give me one brief plank alignment cue."
    }

    val angleSummary = filteredAngles.entries.joinToString { (name, angleDiff) ->
        "$name: $angleDiff"
    }
    return "I'm doing $poseName pose. My joints have the following significant angle differences to the teacher's: $angleSummary"
}
```

The function calls `angleDifference()` with values that ensure it filters out small differences using `JOINT_DIFF_FILTER` and limits the prompt to `MAX_ENTRIES` joint differences.

The fallback prompt handles situations when the learner is already close to the reference pose. The LLM still creates a useful instruction but isn’t forced to invent a large correction from tiny angle changes. This fallback isn’t hit in the full AI Yoga Tutor, as a different “praise” prompt is given when the score is greater than 70.

## Emit prompts from the ViewModel

Open `ui/viewmodels/MainViewModel.kt`.

Replace the TODO in `userPosePrompt` with the following code:

```kotlin
val userPosePrompt: Flow<Pair<String, Double>> = 
    sharedUserPoseResults
        .filter { !_isSpeaking.value }
        .map { userResult -> 
            generateLlmPromptFrom(
                referenceAngles = userResult.referenceAngles,
                userAngles = userResult.userAngles,
                poseName = poseName
            ) to userResult.scoreVal
        }
        .flowOn(Dispatchers.Default)
```

The prompt is paired with the score because fuller app versions might use both values when deciding what feedback is spoken. In this app, the LLM step uses the prompt text.

The `_isSpeaking` filter prevents the app from generating more feedback while the previous correction is still being spoken. You’ll add speech later, but keeping the filter here makes the data flow ready for that final step.

The `flowOn(Dispatchers.Default)` call keeps prompt formatting off the main thread. The work is small, but keeping camera, scoring, prompt generation, LLM inference, and speech on clear execution paths makes the pipeline easier to tune.

## Check the prompt in Logcat

You’ll add the local LLM in the next section. For now, add a temporary collector so you can see the prompt in Logcat.

Open `MainActivity.kt` and update `collectAppState()` so it also collects `userPosePrompt`:

```kotlin
private suspend fun collectAppState() = coroutineScope {
    launch {
        mainViewModel.userPoseScore.collect { score ->
            score.text = getString(R.string.score) + it
        }
    }

    launch {
        mainViewModel.userPosePrompt.collect { prompt ->
            Log.d(TAG, "Pose prompt: ${prompt.first}")
        }
    }
}
```

## Run the app

Build and run the app on your Android device.

Open Logcat and filter for `MainActivity`. As you move in front of the camera, expect short prompts that describe the largest differences from the reference plank pose.

The app now has the complete input side of the tutor pipeline:

```plaintext
Camera frame -> pose landmarks -> angle score -> text prompt
```

## What you’ve accomplished and what’s next

You’ve now turned pose and score data into a compact text prompt for the local LLM.

Next, you’ll add Arm’s AI Chat library and send the prompt to the LLM.