Deploy Arcee AFM-4.5B on Arm-based Google Cloud Axion with Llama.cpp
Introduction
AFM-4.5B deployment on Google Cloud Axion with Llama.cpp
Provision a Google Cloud Axion Arm64 environment
Configure your Google Cloud Axion Arm64 environment
Build Llama.cpp on Google Cloud Axion Arm64
Install Python dependencies for Llama.cpp
Download and optimize the AFM-4.5B model for Llama.cpp
Run inference with AFM-4.5B using Llama.cpp
Benchmark and evaluate AFM-4.5B quantized models on Axion
Review your AFM-4.5B deployment on Axion
Next Steps
Deploy Arcee AFM-4.5B on Arm-based Google Cloud Axion with Llama.cpp
Introduction
AFM-4.5B deployment on Google Cloud Axion with Llama.cpp
Provision a Google Cloud Axion Arm64 environment
Configure your Google Cloud Axion Arm64 environment
Build Llama.cpp on Google Cloud Axion Arm64
Install Python dependencies for Llama.cpp
Download and optimize the AFM-4.5B model for Llama.cpp
Run inference with AFM-4.5B using Llama.cpp
Benchmark and evaluate AFM-4.5B quantized models on Axion
Review your AFM-4.5B deployment on Axion
Next Steps
Who is this for?
This Learning Path is for developers and ML engineers who want to deploy Arcee's AFM-4.5B small language model on Google Cloud Axion instances using Llama.cpp.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Launch an Arm-based Compute Engine instance on Google Cloud Axion
- Build and install Llama.cpp from source
- Download and quantize the AFM-4.5B model from Hugging Face
- Run inference on the quantized model using Llama.cpp
- Evaluate model quality by measuring perplexity
Prerequisites
Before starting, you will need the following:
- A
Google Cloud account
with permission to launch Axion (
c4a-standard-16or larger) instances - Basic familiarity with Linux and SSH
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
llama.cpp. First, you’ll provision an Axion-based VM, prepare the system and Python environment, and build llama.cpp from source. Then, you’ll download and quantize the model, and run inference. You’ll make concrete choices about instance size and storage, manage packages in an isolated environment, and use the upstream llama.cpp repository that includes the required AFM support. By the end, you’ll perform a perplexity run to assess model quality. A successful outcome includes a clean build, a loadable quantized model, token generation during a test prompt, and a reported perplexity value.Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
c4a-standard-16 or larger, as specified. Smaller types are not listed for this workflow.ggerganov/llama.cpp. Arcee AI has contributed the required modeling code upstream, so you don’t need a custom fork.llama.cpp can load. A brief prompt should generate tokens without errors, and a perplexity run should complete with a numeric value.