Deploy a LLM-based Vision Chatbot with PyTorch and Hugging Face Transformers on Google Axion processors

About this Learning Path

Who is this for?

This Learning Path is for software developers and ML engineers who are interested in deploying a production-ready vision chatbot for their application with optimized performance on the Arm Architecture.

What will you learn?

Upon completion of this learning path, you will be able to:

• Build a frontend with Streamlit to input images and prompts.
• Build the backend to download a Llama 3.2-Vision model, quantize it, and run it using PyTorch and Hugging Face Transformers.
• Monitor and analyze inference on Arm CPUs.

Prerequisites

Before starting, you will need the following:

• A Google Cloud Axion compute instance or any Arm-based instance from a cloud service provider with at least 32 cores.
• Familiarity with REST APIs and web services.
• A basic understanding of Python and ML concepts.
• A basic understanding of Streamlit.
• A basic understanding of LLM fundamentals.