Run Phi-3 on Windows on Arm using ONNX Runtime
Introduction
Set up your Environment
Build ONNX Runtime
Build ONNX Runtime Generate() API
Run Phi3 Model
Next Steps
Run Phi-3 on Windows on Arm using ONNX Runtime
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Use the Phi-3 Mini (3.3B) short-context (4K) ONNX model, which is quantized to 4 bits.
Confirm the build completes without errors and produces the expected binaries for WoA. If the build succeeds, proceed to build the
Generate() API and run the model runner.Build the
onnxruntime-genai source and use the provided runner to generate text. Successful text generation indicates the Generate() API is correctly integrated.Expect generated text from the Phi-3 Mini model along with performance metrics printed by the runner. If output appears without errors, continue to experiment with prompts.
No. The path builds and runs ONNX Runtime with KleidiAI acceleration, and it doesn’t list any additional KleidiAI configuration.