Build a Sentiment-Aware Voice Assistant with On-Device LLMs

About this Learning Path

Who is this for?

This Learning Path is for developers, ML practitioners, and game developers interested in building on-device AI applications, including voice interfaces, real-time interactions with non-player characters (NPCs), and edge AI systems powered by LLMs on Arm platforms.

What will you learn?

Upon completion of this Learning Path, you will be able to:

• Build a voice-to-LLM pipeline using Whisper and llama.cpp.
• Train a voice sentiment classification model using HuBERT on the RAVDESS dataset.
• Quantize the model and convert into ONNX Runtime for on-device inference.
• Integrate sentiment classification model with voice-to-LLM pipeline to generate context-aware LLM responses.

Prerequisites

Before starting, you will need the following:

• Python 3.9 or later for programming.
• A working microphone for voice input.
• Basic Python and command-line knowledge.