# Build an offline voice chatbot with faster-whisper and vLLM on DGX Spark

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/)
- [Build an offline voice assistant with whisper and vLLM](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/1_offline_voice_assistant/)
- [Install faster-whisper for local speech recognition](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/2_setup/)
- [Build a real-time STT pipeline on CPU](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/3_fasterwhisper/)
- [Fine-tune segmentation parameters](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/3a_segmentation/)
- [Build a real-time offline voice chatbot using STT and vLLM](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/4_vllm/)
- [Connect speech recognition to vLLM for real-time voice interaction](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/4a_integration/)
- [Specialize offline voice assistants for customer service](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/5_chatbot_prompt/)
- [Enable context-aware dialogue with short-term memory](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/6_chatbot_contextaware/)
- [Next Steps](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_voicechatbot/_next-steps/)

## About this Learning Path

|  |  |
| --- | --- |
| Skill level: | Advanced |
| Reading time: | 1 hr |
| Last updated: | 29 Jul 2026 |

|  |  |
| --- | --- |
| Author: | Odin Shen , Arm<br>[GitHub](https://github.com/odincodeshen)<br>[LinkedIn](https://linkedin.com/in/odin-shen-lmshen) |
| Arm IP: | [Neoverse](https://support.arm.com/?tab=compute-ip&Product%20Type=Infrastructure%20Processors) |
| Tags: | [Performance and Architecture](https://learn.arm.com/tag/performance-and-architecture) [Linux](https://learn.arm.com/tag/linux) [Docker](https://learn.arm.com/tag/docker) [Python](https://learn.arm.com/tag/python) |

### Who is this for?
This is an advanced topic for developers and ML engineers who want to build private, offline voice assistant systems on Arm-based servers such as DGX Spark.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Explain the architecture of an offline voice chatbot pipeline combining speech-to-text (STT) and vLLM
- Capture and segment real-time audio using PyAudio and Voice Activity Detection (VAD)
- Transcribe speech using faster-whisper and generate replies using vLLM
- Tune segmentation and prompt strategies to improve latency and response quality
- Deploy and run the full pipeline on Arm-based systems such as DGX Spark

### Prerequisites
Before starting, you'll need the following prerequisites:
- An NVIDIA DGX Spark system with at least 15 GB of available disk space
- A USB microphone for audio input

### Summary
You’ll build a local voice chatbot on an Arm-based NVIDIA DGX Spark with `faster-whisper` for speech-to-text and vLLM for response generation. You’ll capture microphone audio with PyAudio, add voice and turn detection, tune segmentation for stable low-latency transcription, and connect the CPU STT pipeline to GPU-backed vLLM. You’ll validate segmented transcripts followed by local replies.

### Frequently asked questions

**What result should I expect after installing `faster-whisper`?**
After a successful installation, you can transcribe a short audio sample or live microphone input with readable text and no runtime errors. Use this to confirm the installation before moving on to pipeline changes.

**When should I upgrade the speech model in the CPU STT pipeline?**
Upgrade after you confirm baseline transcription works. The build step adds a more accurate model and VAD. If latency increases, proceed to segmentation tuning.

**How do I know VAD and turn detection are working correctly?**
Transcriptions should arrive as sentence-like chunks, and pauses should start new segments. If long monologues merge into one block or speech is cut mid-sentence, adjust the segmentation parameters.

**What should I verify before integrating vLLM with the STT engine?**
Ensure the CPU-based STT runs in real time on your DGX Spark and produces stable, segmented text. A clean, timely text stream simplifies downstream integration with vLLM.

**What behavior confirms the end-to-end offline chatbot is running?**
Speak into the microphone and watch for segmented transcriptions from `faster-whisper`, followed by a locally generated reply from vLLM. Seeing this sequence consistently indicates the pipeline is integrated and running on the system.