Another way to run the model is to use an Android GUI app. You can use the Android demo application included in the onnxruntime-inference-examples repository to demonstrate local inference.
git clone https://github.com/microsoft/onnxruntime-inference-examples
cd onnxruntime-inference-examples
git checkout 7a635daae48450ff142e5c0848a564b245f04112
You could probably use a later commit but these steps have been tested with the commit 7a635daae48450ff142e5c0848a564b245f04112
.
Open the mobile\examples\phi-3\android
directory with Android Studio.
Copy ONNX Runtime AAR and ONNX Runtime GenAI AAR you built earlier in this learning path:
Copy onnxruntime\build\Windows\Release\java\build\android\outputs\aar\onnxruntime-release.aar onnxruntime-inference-examples\mobile\examples\phi-3\android\app\libs\
Copy onnxruntime-genai\build\Android\Release\src\java\build\android\outputs\aar\onnxruntime-genai-release.aar onnxruntime-inference-examples\mobile\examples\phi-3\android\app\libs\
Update build.gradle.kts (:app)
as below:
// ONNX Runtime with GenAI
//implementation("com.microsoft.onnxruntime:onnxruntime-android:latest.release")
implementation(files("libs/onnxruntime-release.aar"))
//implementation(files("libs/onnxruntime-genai-android-0.8.1.aar"))
implementation(files("libs/onnxruntime-genai-release.aar"))
Finally, click File > Sync Project with Gradle
When you select Run, the build will be executed, and then the app will be copied and installed on the Android device. This app will automatically download the Phi-3-mini model during the first run. After the download, you can input the prompt in the text box and execute it to run the model.
You should now see a running app on your phone, which looks like this: