Deploy Phi-4-mini model with ONNX Runtime on Azure Cobalt 100
Introduction
Demo
Build ONNX Runtime and set up the Phi-4-mini Model
Run the Chatbot Server
Interact with the Phi-4-mini Chatbot
Next Steps
Deploy Phi-4-mini model with ONNX Runtime on Azure Cobalt 100
Who is this for?
This is an advanced topic for developers, ML engineers, and cloud practitioners looking to deploy Microsoft's Phi Models on Arm-based servers using ONNX Runtime.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Quantize and run the Phi-4-mini model with ONNX Runtime on Azure.
- Analyze performance on Arm Neoverse N2 based Azure Cobalt 100 VMs.
Prerequisites
Before starting, you will need the following:
- An Arm-based instance from an appropriate cloud service provider. This Learning Path has been tested on an Azure Cobalt 100 virtual machine.
- Basic understanding of Python and machine learning concepts.
- Familiarity with ONNX Runtime and Azure cloud services.
- Knowledge of Large Language Model (LLM) fundamentals.