Accelerate multimodal Voice Assistant performance with KleidiAI and SME2
Introduction
Set up your environment
Overview
Build the Voice Assistant
Run the Voice Assistant
KleidiAI
Next Steps
Accelerate multimodal Voice Assistant performance with KleidiAI and SME2
Switch to developer mode
In the previous section, you built the Voice Assistant application. Now, you’ll install it on your Android phone. The easiest way is to enable developer mode and use a USB cable to upload the application.
By default, Android devices ship with developer mode disabled. To enable it, follow these instructions .
Install the Voice Assistant
Once developer mode is enabled, connect your phone to your computer with USB. It should appear as a running device in the top toolbar. Select the device and click Run (a small green triangle, as shown below). This transfers the app to your phone and launches it.
In the graphic below, a Google Pixel 8 Pro phone is connected to the USB cable:
Upload the Voice App
Launch the Voice Assistant
The app starts with this welcome screen:
Welcome Screen
Tap Press to talk at the bottom of the screen to begin speaking your request.
Voice Assistant controls
You can use application controls to enable extra functionality or gather performance data.
| Button | Control name | Description |
|---|---|---|
| 1 | Performance counters | Performance counters are hidden by default, click this to show speech recognition time, LLM encode and decode rate. |
| 2 | Speech generation | Speech generation is disabled by default, click this to use Android Text-to-Speech and get audible answers. |
| 3 | Reset conversation | By default, the application keeps context so you can follow-up questions, click this to reset voice assistant conversation history. |
Click the icon circled in red in the top left corner to show or hide these metrics:
Performance Counters
Multimodal Question Answering
If you have built the application using the default llama.cpp backend, you can also use it in multimodal (input + text) question answering mode.
For this, click the image button first:
Add image button
This will bring up the photos you can chose from:
Choose image from the gallery
Choose the image, and add image for voice assistant:
Add image to the question
You can now ask questions related to this image, the large language model will you the image and text for multimodal question answering.
Add image to the question
Now that you have explored how the android application is set up and built, you can see in detail how KleidiAI library is used in the next step.