Give feedback

How would you rate this Learning Path?

Review frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Which EC2 instance type and storage size should I choose when launching?
Use a c8g.4xlarge (or larger) EC2 instance powered by Graviton4 and allocate at least 128 GB of available storage. Configure the volume size during instance launch in the EC2 service so the space is available before you build and run the model.
When should I create the SSH key pair and what is it used for?
Create the SSH key pair in the AWS Management Console before launching the instance. You will use it to connect to the Graviton4 host over SSH for setup, building, and running inference.
Which `llama.cpp` repository should I clone, and why build from source here?
Clone the upstream repository at https://github.com/ggerganov/llama.cpp as shown in the steps. Build from source because AFM-4.5B support was contributed upstream, so the standard repo contains the necessary modeling code.
Do I need a Python virtual environment for this workflow?
Yes. Creating a virtual environment, such as env-llama-cpp, isolates the Python interpreter and packages required for the model workflow and avoids conflicts with system packages.
How do I know that quantization and perplexity evaluation worked?
After downloading AFM-4.5B and completing quantization, run the llama.cpp steps that load the quantized model and perform evaluation. Expect the program to load the model without errors and print a perplexity value you can use to compare quality across runs.

Share

Tell others about this Learning Path.

Back
Return to Servers and Cloud Computing