Deploy DeepSeek-R1 on Arm Servers with llama.cpp
Introduction
Run a DeepSeek-R1 chatbot on Arm servers
Access the chatbot using the OpenAI-compatible API
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  30 min
Last updated:  31 Jul 2026
Skill level:
  Introductory
Reading time:
  30 min
Last updated:
  31 Jul 2026
Author:Tianyu Li, Arm
Arm IP:Neoverse
Tags: ML AWS Microsoft Azure Google Cloud Oracle Linux LLM Generative AI Python
Author:
  • Tianyu Li, Arm
Arm IP:
Neoverse
Tags:
ML AWS Microsoft Azure Google Cloud Oracle Linux LLM Generative AI Python

Who is this for?

This Learning Path is for developers who want to run DeepSeek-R1 on Arm-based servers.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Clone and build llama.cpp on your Arm-based server.
  • Download a pre-quantized DeepSeek-R1 model from Hugging Face.
  • Run the model on your Arm CPU and benchmark its performance.

Prerequisites

Before starting, you will need the following:

  • An Arm-based instance from a cloud provider or an on-premise Arm server. This Learning Path was tested on an AWS Graviton4 r8g.24xlarge instance.

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build llama.cpp on an Arm-based Ubuntu server, download a pre-quantized DeepSeek-R1 model from Hugging Face, and run it for CPU inference. You’ll launch a persistent llama.cpp server with an OpenAI-compatible API, deploy a chatbot with the DeepSeek-R1 671B LLM, and send requests from the same or another machine. You’ll then confirm the setup with API responses and basic benchmarks.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
What do I need before running the 671B model?
Use an Arm server running Ubuntu 24.04 LTS with at least 64 cores, 512 GB of RAM, and 400 GB of disk space. The Learning Path was tested on an AWS Graviton4 r8g.24xlarge instance.
Which DeepSeek-R1 model should I download?
Download a pre-quantized DeepSeek-R1 model from Hugging Face. Quantization enables efficient CPU inference with llama.cpp.
How do I know the llama.cpp server binary is available?
Running make during the build step creates the server executable. If it’s missing, repeat the build step before starting the server.
How do I access the model repeatedly without restarting it?
Start the llama.cpp server and use its OpenAI-compatible API to submit multiple requests. You can also send requests from another machine to the host running the server.
Why do I install jq?
The examples use jq to work with JSON. jq helps format and parse responses returned by the OpenAI-compatible API calls.
Next