Run Phi-3 on Windows on Arm using ONNX Runtime
Introduction
Set up your Environment
Build ONNX Runtime
Build ONNX Runtime Generate() API
Run Phi3 Model
Next Steps
Run Phi-3 on Windows on Arm using ONNX Runtime
Who is this for?
This is an advanced topic for developers looking to build ONNX Runtime for Windows on Arm (WoA) and leverage the Generate() API to run Phi-3 inference with KleidiAI acceleration.
What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Build ONNX Runtime and enable the Generate() API for Windows on Arm.
- Run inference with a Phi-3 model using ONNX Runtime with KleidiAI acceleration.
Prerequisites
Before starting, you'll need the following prerequisites:
- A Windows on Arm computer such as a Lenovo Thinkpad X13 running Windows 11, or a Windows on Arm virtual machine .
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You’ll build ONNX Runtime and the
Generate() API from source on Windows on Arm, then run Phi-3 Mini inference with KleidiAI acceleration. First, you’ll configure the development environment, compile the inference engine, and add the generation loop. Then, you’ll download the short-context ONNX model and run it to produce text and performance metrics.