Deploy Arcee AFM-4.5B on Arm-based Google Cloud Axion with Llama.cpp
Introduction
AFM-4.5B deployment on Google Cloud Axion with Llama.cpp
Provision a Google Cloud Axion Arm64 environment
Configure your Google Cloud Axion Arm64 environment
Build Llama.cpp on Google Cloud Axion Arm64
Install Python dependencies for Llama.cpp
Download and optimize the AFM-4.5B model for Llama.cpp
Run inference with AFM-4.5B using Llama.cpp
Benchmark and evaluate AFM-4.5B quantized models on Axion
Review your AFM-4.5B deployment on Axion
Next Steps
Deploy Arcee AFM-4.5B on Arm-based Google Cloud Axion with Llama.cpp
Introduction
AFM-4.5B deployment on Google Cloud Axion with Llama.cpp
Provision a Google Cloud Axion Arm64 environment
Configure your Google Cloud Axion Arm64 environment
Build Llama.cpp on Google Cloud Axion Arm64
Install Python dependencies for Llama.cpp
Download and optimize the AFM-4.5B model for Llama.cpp
Run inference with AFM-4.5B using Llama.cpp
Benchmark and evaluate AFM-4.5B quantized models on Axion
Review your AFM-4.5B deployment on Axion
Next Steps
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Use
c4a-standard-16 or larger, as specified. Smaller types are not listed for this workflow.Confirm that your project has quota for Axion instances and that sufficient storage is available. The Learning Path calls for at least 128 GB to accommodate the model and dependencies.
Clone the standard upstream repository from
ggerganov/llama.cpp. Arcee AI has contributed the required modeling code upstream, so you don’t need a custom fork.The build should finish without errors and produce binaries that run as shown in the steps. Invoking the tool to display its help or version is a quick validation before proceeding.
Quantization produces a smaller model artifact that
llama.cpp can load. A brief prompt should generate tokens without errors, and a perplexity run should complete with a numeric value.