Build an offline voice chatbot with faster-whisper and vLLM on DGX Spark
Introduction
Build an offline voice assistant with whisper and vLLM
Install faster-whisper for local speech recognition
Build a real-time STT pipeline on CPU
Fine-tune segmentation parameters
Build a real-time offline voice chatbot using STT and vLLM
Connect speech recognition to vLLM for real-time voice interaction
Specialize offline voice assistants for customer service
Enable context-aware dialogue with short-term memory
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr
Last updated:  13 Feb 2026
Skill level:
  Advanced
Reading time:
  1 hr
Last updated:
  13 Feb 2026
Author:Odin Shen, Arm
Arm IP:Neoverse
Tags: Performance and Architecture Linux Docker Python
Author:
  • Odin Shen, Arm
Arm IP:
Neoverse
Tags:
Performance and Architecture Linux Docker Python

Who is this for?

This is an advanced topic for developers and ML engineers who want to build private, offline voice assistant systems on Arm-based servers such as DGX Spark.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Explain the architecture of an offline voice chatbot pipeline combining speech-to-text (STT) and vLLM
  • Capture and segment real-time audio using PyAudio and Voice Activity Detection (VAD)
  • Transcribe speech using faster-whisper and generate replies using vLLM
  • Tune segmentation and prompt strategies to improve latency and response quality
  • Deploy and run the full pipeline on Arm-based systems such as DGX Spark

Prerequisites

Before starting, you will need the following:

  • An NVIDIA DGX Spark system with at least 15 GB of available disk space
  • A USB microphone for audio input
Next