Who is this for?

This Learning Path is for developers and ML engineers who want to deploy Arcee's AFM-4.5B small language model on AWS Graviton4 instances using Llama.cpp.

What will you learn?

After completing this Learning Path, you'll be able to perform the following tasks:

  • Launch an Arm-based EC2 instance on AWS Graviton4
  • Build and install Llama.cpp from source
  • Download and quantize the AFM-4.5B model from Hugging Face
  • Run inference on the quantized model using Llama.cpp
  • Evaluate model quality by measuring perplexity

Prerequisites

Before starting, you'll need the following prerequisites:

  • An AWS account with permission to launch Graviton4 (c8g.4xlarge or larger) instances
  • Basic familiarity with Linux and SSH

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll provision an Arm-based Amazon EC2 instance powered by AWS Graviton4, prepare a Linux environment, and build the llama.cpp inference engine from source. With a dedicated Python virtual environment, you’ll retrieve Arcee’s AFM-4.5B model from Hugging Face, perform quantization, and run inference using llama.cpp on Graviton4. You’ll learn to make practical choices, including instance sizing, storage, dependency setup, and repository selection, so the model compiles and executes cleanly on Arm. By the end, you’ll perform a perplexity-based quality check to validate that the quantized model runs, and interpret the reported perplexity as an objective signal of model quality.
Next