Run Phi-3 on Windows on Arm using ONNX Runtime
Introduction
Set up your Environment
Build ONNX Runtime
Build ONNX Runtime Generate() API
Run Phi3 Model
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr
Last updated:  01 May 2025
Skill level:
  Advanced
Reading time:
  1 hr
Last updated:
  01 May 2025
Author:Barbara Corriero, Arm
Arm IP:Cortex-A
Tags: ML Windows Visual Studio C++ Python Git cmake ONNX Runtime
Author:
  • Barbara Corriero, Arm
Arm IP:
Cortex-A
Tags:
ML Windows Visual Studio C++ Python Git cmake ONNX Runtime

Who is this for?

This is an advanced topic for developers looking to build ONNX Runtime for Windows on Arm (WoA) and leverage the Generate() API to run Phi-3 inference with KleidiAI acceleration.

What will you learn?

Upon completion of this learning path, you will be able to:

  • Build ONNX Runtime and enable the Generate() API for Windows on Arm.
  • Run inference with a Phi-3 model using ONNX Runtime with KleidiAI acceleration.

Prerequisites

Before starting, you will need the following:

  • A Windows on Arm computer such as a Lenovo Thinkpad X13 running Windows 11, or a Windows on Arm virtual machine .
Next