Build and Run vLLM on Arm Servers
Introduction
Build a vLLM from Source Code
Run batch inference using vLLM
Run an OpenAI-compatible server
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  45 min
Last updated:  23 Feb 2026
Skill level:
  Introductory
Reading time:
  45 min
Last updated:
  23 Feb 2026
Author:Jason Andrews, Arm
Arm IP:Neoverse
Tags: ML AWS Microsoft Azure Google Cloud Oracle Linux vLLM LLM Generative AI Python Hugging Face
Author:
  • Jason Andrews, Arm
Arm IP:
Neoverse
Tags:
ML AWS Microsoft Azure Google Cloud Oracle Linux vLLM LLM Generative AI Python Hugging Face

Who is this for?

This is an introductory topic for software developers and AI engineers interested in learning how to use the vLLM library on Arm servers.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Build vLLM from source on an Arm server.
  • Download a Qwen LLM from Hugging Face.
  • Run local batch inference using vLLM.
  • Create and interact with an OpenAI-compatible server provided by vLLM on your Arm server.

Prerequisites

Before starting, you will need the following:

  • An Arm-based instance from a cloud service provider, or a local Arm Linux computer with at least 8 CPUs and 16 GB RAM.
Next