Who is this for?

This Learning Path is for developers and ML engineers who want to deploy Arcee's AFM-4.5B small language model on Google Cloud Axion instances using Llama.cpp.

What will you learn?

After completing this Learning Path, you'll be able to perform the following tasks:

  • Launch an Arm-based Compute Engine instance on Google Cloud Axion
  • Build and install Llama.cpp from source
  • Download and quantize the AFM-4.5B model from Hugging Face
  • Run inference on the quantized model using Llama.cpp
  • Evaluate model quality by measuring perplexity

Prerequisites

Before starting, you'll need the following prerequisites:

  • A Google Cloud account with permission to launch Axion (c4a-standard-16 or larger) instances
  • Basic familiarity with Linux and SSH

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll deploy Arcee’s AFM-4.5B model on Arm-based Google Cloud Axion using llama.cpp. First, you’ll provision an Axion-based VM, prepare the system and Python environment, and build llama.cpp from source. Then, you’ll download and quantize the model, and run inference. You’ll make concrete choices about instance size and storage, manage packages in an isolated environment, and use the upstream llama.cpp repository that includes the required AFM support. By the end, you’ll perform a perplexity run to assess model quality. A successful outcome includes a clean build, a loadable quantized model, token generation during a test prompt, and a reported perplexity value.
Next