Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark
Introduction
Explore Grace Blackwell architecture for efficient quantized LLM inference
Verify your Grace Blackwell system readiness for AI inference
Build the GPU version of llama.cpp on GB10
Build the CPU version of llama.cpp on GB10
Analyze CPU instruction mix using Process Watch
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  1 hr
Last updated:  29 Jul 2026
Skill level:
  Introductory
Reading time:
  1 hr
Last updated:
  29 Jul 2026
Author:Odin Shen, Arm
Arm IP:Cortex-A Cortex-X
Tags: ML Linux Python C Bash llama.cpp
Author:
  • Odin Shen, Arm
Arm IP:
Cortex-A Cortex-X
Tags:
ML Linux Python C Bash llama.cpp

Who is this for?

This is an introductory topic for AI practitioners, performance engineers, and system architects who want to learn how to deploy and optimize quantized large language models (LLMs) on NVIDIA DGX Spark systems powered by the Grace-Blackwell (GB10) architecture.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Describe the Grace–Blackwell (GB10) architecture and its support for efficient AI inference
  • Build CUDA-enabled and CPU-only versions of llama.cpp for flexible deployment
  • Validate the functionality of both builds on the DGX Spark platform
  • Analyze how Armv9 SIMD instructions accelerate quantized LLM inference on the Grace CPU

Prerequisites

Before starting, you will need the following:

  • Access to an NVIDIA DGX Spark system with at least 15 GB of available disk space
  • Familiarity with command-line interfaces and basic Linux operations
  • Understanding of CUDA programming basics, as well as GPU and CPU compute concepts
  • Basic knowledge of quantized large language models (LLMs) and machine learning inference
  • Experience building software from source using CMake and make

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll prepare NVIDIA DGX Spark with its Grace CPU and Blackwell GPU, build llama.cpp for CUDA and CPU execution, and inspect Armv9 vector instructions during quantized LLM inference. You’ll verify CUDA, compile both variants, and use Process Watch to examine Neon activity and the current lack of SVE and SVE2. You’ll finish with GPU and CPU binaries.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I know my DGX Spark is ready before building?
Confirm the Grace CPU configuration, operating system, Blackwell GPU, and CUDA drivers are active, and verify the CUDA 13 toolkit is installed.
Which build should I start with, GPU-enabled or CPU-only?
If the Blackwell GPU and CUDA are available, start with the GPU-enabled llama.cpp build. Then, build the CPU-only version to run on the Grace CPU and to keep a flexible deployment option.
What result should I expect after a successful build?
The build produces a compiled llama.cpp binary targeting either the GPU or the CPU that runs quantized LLM inference. A quick test run should complete without errors on the DGX Spark.
How do I validate that the CPU-only build uses Armv9 vector features?
Run an inference with the CPU-only binary and analyze it with Process Watch to observe the instruction mix. Expect to see Neon SIMD activity; SVE and SVE2 might remain inactive under the current kernel configuration.
Why don’t I see SVE or SVE2 instructions in my Process Watch results?
Although the Grace CPU supports SVE and SVE2, the current kernel exposes a fixed 16-byte (128-bit) SVE vector length. The llama.cpp workload therefore uses Neon instructions instead. Future kernel updates might enable SVE2 instructions.
Next