Deploy models with the Arm AI Portal
Introduction
Browse and compare models in the Arm AI Portal
Deploy a TinyLlama model from the Arm AI Portal
Deploy an ExecuTorch image classifier using Topo
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  30 min
Last updated:  07 Sep 2026
Skill level:
  Introductory
Reading time:
  30 min
Last updated:
  07 Sep 2026
Author:Andrew Pickard
Arm IP:Cortex-A Neoverse
Tags: ML Linux Python Docker
Author:
  • Andrew Pickard
Arm IP:
Cortex-A Neoverse
Tags:
ML Linux Python Docker

Who is this for?

This is an introductory topic for developers who want to deploy optimized models from the Arm AI Portal.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Discover how AI Portal helps you select a suitable model
  • Deploy a model using instructions on its Hugging Face page
  • Deploy a model using Topo, Arm's deployment tool

Prerequisites

Before starting, you will need the following:

  • An Arm Linux target device to deploy models on, with enough memory and storage for the selected model and application
  • A development host running Windows, macOS, or Linux with internet access and permission to install tools
  • Python 3 with pip and virtual environment support
  • Docker and Topo to run containerized code examples from the AI Portal
  • SSH access and permission to provision keys when deploying to a remote target
  • A Hugging Face account for downloading models
  • Basic familiarity with terminal commands and containers

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll use the Arm AI Portal to compare models with filters and analytics, then follow two deployment workflows. First, you’ll download the TinyLlama-1.1B-Chat INT4 — ONNX GenAI (Graviton G4) model from Hugging Face and run it on an Arm Linux target with ONNX Runtime. Then, you’ll deploy a containerized ExecuTorch image classifier code example with Topo.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I narrow down models on AI Portal?
Use the Arm AI Portal’s filters and analytics to compare models. Open a model’s details page to review related content, such as Learning Paths and code examples.
How do I deploy and test TinyLlama on an Arm Linux target?
Install the required Python packages, create a Hugging Face access token, and set it in HF_TOKEN. Use snapshot_download to download the model files, link them from model_dir, and run the supplied ONNX Runtime test script.
What do I need on the Arm Linux target to start the TinyLlama deployment?
Update the packages and install Python tooling with sudo apt update and sudo apt install -y python3-pip python3-venv. The model is optimized for AWS Graviton 4-based Amazon EC2 instances, such as M8g.
What should I check if the deployed image classifier URL is inaccessible?
If the deployment succeeded on a cloud instance, make sure that you’ve enabled access to port 7860. If deployment failed, check the target’s available disk space and confirm that the Docker container is running with docker container ls.
How do I free port 7860 before deploying another code example?
Stop the containers that expose port 7860 with docker container stop $(docker container ls --filter expose=7860 -q). You can then deploy another code example that uses the port.
Next