# Run Phi3 Model

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/)
- [Set up your Environment](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/1-dev-env-setup/)
- [Build ONNX Runtime](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/2-build-onnxruntime/)
- [Build ONNX Runtime Generate() API](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/3-build-onnxruntime-generate-api/)
- [Run Phi3 Model](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/4-run-benchmark-on-woa/)
- [Next Steps](https://learn.arm.com/learning-paths/laptops-and-desktops/win_on_arm_build_onnxruntime/_next-steps/)

## Run the Phi-3 model on your Windows on Arm machine
In this section, you’ll download the Phi-3 Mini model and run it on your WoA machine - either physical or virtual. You’ll use a simple model runner that also reports performance metrics.

The Phi-3 Mini (3.3B) model is available in two versions:
- Short context (4K) - supports shorter prompts and uses less memory.
- Long context (128K) - supports longer prompts and outputs but consumes more memory.

This Learning Path uses the short context version, which is quantized to 4-bits.

The Phi-3 Mini model used here is in ONNX format.

### Set up
[Phi-3 ONNX models](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx) are hosted on HuggingFace. Hugging Face uses Git for both version control and to download the ONNX model files, which are large.

### Install Git LFS
You’ll first need to install the Git Large File Storage (LFS) extension:
```bash
winget install -e --id GitHub.GitLFS
git lfs install
```
If you don’t have winget, [download the installer manually](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=windows).

If Git LFS is already installed, you’ll see `Git LFS initialized`.

### Install Hugging Face CLI
You then need to install the `HuggingFace CLI`:
```bash
pip install huggingface-hub[cli]
```

### Download the Phi-3-Mini (4K) model
```bash
cd C:\Users\%USERNAME%
cd repos\lp
huggingface-cli download microsoft/Phi-3-mini-4k-instruct-onnx --include cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/* --local-dir .
```
This command downloads the model into a folder named `cpu_and_mobile`.

### Build the Model Runner (ONNX Runtime GenAI C Example)
In the previous step, you built the ONNX Runtime Generate() API from source. Now, copy over the resulting headers and Dynamically Linked Libraries into the appropriate folders (`lib` and `include`).

Building from source is a better practice because the examples usually are updated to run with the latest changes:
```bash
copy onnxruntime\build\Windows\Release\Release\onnxruntime.* onnxruntime-genai\examples\c\lib
cd onnxruntime-genai
copy build\Windows\Release\Release\onnxruntime-genai.* examples\c\lib
copy src\ort_genai.h examples\c\include\
copy src\ort_genai_c.h examples\c\include\
```
You can now build the model runner executable in the ‘onnxruntime-genai’ folder using the commands below:
```bash
cd examples/c
cmake -A arm64 -S . -B build -DPHI3=ON
cd build
cmake --build . --config Release
```
After a successful build, the binary `phi3` will be created in the ‘onnxruntime-genai’ folder:
```bash
__output__ dir Release\phi3.exe
```

### Run the model
Execute the model using the following command:
```bash
cd C:\Users\%USERNAME%
cd repos\lp
.\onnxruntime-genai\examples\c\build\Release\phi3.exe .\cpu_and_mobile\cpu-int4-rtn-block-32-acc-level-4 cpu
```
This will allow the runner program to load the model. It will then prompt you to input the text prompt to the model as shown:
```plaintext
__output__ -------------
__output__ Hello, Phi-3!
__output__ -------------
__output__ C++ API
__output__ Creating config...
__output__ Creating model...
__output__ Creating tokenizer...
__output__ Prompt: (Use quit() to exit) Or (To terminate current output generation, press Ctrl+C)
```
After you enter your input prompt, the text output by the model will be displayed. On completion, performance metrics similar to those shown below should be displayed:
```plaintext
Prompt length: 64, New tokens: 931, Time to first: 1.79s, Prompt tokens per second: 35.74 tps, New tokens per second: 6.34 tps
```
You have successfully run the Phi-3 model on your Windows on Arm device.