Run Phi-3 on Windows on Arm using ONNX Runtime
Introduction
Set up your Environment
Build ONNX Runtime
Build ONNX Runtime Generate() API
Run Phi3 Model
Next Steps
Run Phi-3 on Windows on Arm using ONNX Runtime
Who is this for?
This is an advanced topic for developers looking to build ONNX Runtime for Windows on Arm (WoA) and leverage the Generate() API to run Phi-3 inference with KleidiAI acceleration.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Build ONNX Runtime and enable the Generate() API for Windows on Arm.
- Run inference with a Phi-3 model using ONNX Runtime with KleidiAI acceleration.
Prerequisites
Before starting, you will need the following:
- A Windows on Arm computer such as a Lenovo Thinkpad X13 running Windows 11, or a Windows on Arm virtual machine .
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Generate() API from source on Windows on Arm, then run Phi-3 Mini inference with KleidiAI acceleration. First, you’ll configure the development environment, compile the inference engine, and add the generation loop. Then, you’ll download the short-context ONNX model and run it to produce text and performance metrics.Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Generate() API and run the model runner.onnxruntime-genai source and use the provided runner to generate text. Successful text generation indicates the Generate() API is correctly integrated.