Deploy a PyTorch vision chatbot on Google Axion
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Open the browser to the app. You’ll see the title LLM Vision Chatbot on Arm with controls to upload an image and enter a prompt. After submitting, the page will display a generated text response that uses the image as context.
Use
http://[your instance ip]:8501 in your browser. If the page doesn’t load, allow inbound TCP traffic to port 8501 in your instance’s security rules.Start the backend script in one terminal with the virtual environment activated. Then, open a new terminal, activate the same environment, and start the Streamlit frontend.
Confirm that
backend.py is running on port 5000 and that frontend.py uses http://localhost:5000/v1/chat/completions. For remote browser access, open the Streamlit frontend on port 8501. The frontend connects to the backend locally.Start
backend.py and wait for the Flask output showing that the server is running on port 5000. The backend.py startup loads the Llama 3.2 Vision model and performs quantization before serving requests.