Deploy Phi-3.5 Vision with ONNX Runtime on Azure Cobalt 100

About this Learning Path

Who is this for?

This is an advanced topic for developers, ML engineers, and cloud practitioners looking to deploy Microsoft's Phi Models on Arm-based servers using ONNX Runtime.

What will you learn?

Upon completion of this learning path, you will be able to:

• Quantize and run the Phi-3.5 vision model with ONNX Runtime on Azure.
• Analyze performance on Arm Neoverse-N2 based Azure Cobalt 100 VMs.

Prerequisites

Before starting, you will need the following:

• An Arm-based instance from an appropriate cloud service provider. This Learning Path has been tested on an Azure Cobalt 100 virtual machine.
• Basic understanding of Python and machine learning concepts.
• Familiarity with ONNX Runtime and Azure cloud services.
• Knowledge of Large Language Model (LLM) fundamentals.