Build a Sentiment-Aware Voice Assistant with On-Device LLMs
Introduction
Understand voice sentiment analysis for on-device AI
Set up your environment
Build the voice-to-LLM pipeline
Train the voice sentiment classification model
Convert and quantize the model
Integrate the voice sentiment classification model
Next Steps
Build a Sentiment-Aware Voice Assistant with On-Device LLMs
Who is this for?
This Learning Path is for developers, ML practitioners, and game developers interested in building on-device AI applications, including voice interfaces, real-time interactions with non-player characters (NPCs), and edge AI systems powered by LLMs on Arm platforms.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Build a voice-to-LLM pipeline using Whisper and llama.cpp.
- Train a voice sentiment classification model using HuBERT on the RAVDESS dataset.
- Quantize the model and convert into ONNX Runtime for on-device inference.
- Integrate sentiment classification model with voice-to-LLM pipeline to generate context-aware LLM responses.
Prerequisites
Before starting, you will need the following:
- Python 3.9 or later for programming.
- A working microphone for voice input.
- Basic Python and command-line knowledge.