Build a RAG pipeline on Arm-based NVIDIA DGX Spark
Introduction
Explore building a RAG pipeline on Arm-based Grace–Blackwell systems
Configure the RAG development environment and models
Add documents to the RAG vector database
Build and run the RAG pipeline
Monitor unified memory performance
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr
Last updated:  28 Jul 2026
Skill level:
  Advanced
Reading time:
  1 hr
Last updated:
  28 Jul 2026
Author:Odin Shen, Arm
Arm IP:Cortex-A
Tags: ML Linux Python llama.cpp Hugging Face
Author:
  • Odin Shen, Arm
Arm IP:
Cortex-A
Tags:
ML Linux Python llama.cpp Hugging Face

Who is this for?

This is an advanced topic for developers who want to build a Retrieval-Augmented Generation (RAG) pipeline on the NVIDIA DGX Spark platform. You'll learn how Arm-based Grace CPUs handle document retrieval and orchestration, while Blackwell GPUs speed up large language model inference using the open-source llama.cpp REST server. This is a great fit if you're interested in combining Arm CPU management with GPU-accelerated AI workloads.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Describe how a RAG system combines document retrieval and language model generation
  • Deploy a hybrid CPU-GPU RAG pipeline on the GB10 platform using open-source tools
  • Use the llama.cpp REST Server for GPU-accelerated inference with CPU-managed retrieval
  • Build a reproducible RAG application that demonstrates efficient hybrid computing

Prerequisites

Before starting, you will need the following:

  • An NVIDIA DGX Spark system with at least 15 GB of available disk space

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build a Retrieval-Augmented Generation (RAG) pipeline on an Arm-based NVIDIA DGX Spark. You’ll prepare embedding and Llama 3.1 8B Instruct models, clean and chunk a sample corpus, create a FAISS index on the Grace CPU, and connect retrieval to GPU inference through the llama.cpp REST server. You’ll run queries and monitor unified memory during active RAG requests.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Do I need to complete the quantized LLM Learning Path before starting this one?
Complete the Learning Path Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark first. That background covers CPU and GPU builds of llama.cpp used in this RAG solution.
Which models should I use for embeddings and generation in this RAG pipeline?
Use e5-base-v2 for embeddings and Llama 3.1 8B Instruct for generation. These models are prepared during the environment setup steps.
What should I expect after preparing documents and building the FAISS index?
The workflow converts your documents into clean, chunked text segments, then vectorizes and indexes them with FAISS. The result is a searchable vector database that returns the most relevant chunks for a query.
How do I know the retrieval and generation components are integrated correctly?
Run a query and confirm the response reflects information from your loaded documentation. Retrieval and indexing run on the Arm Grace CPUs, and generation uses the llama.cpp REST server on Blackwell GPUs.
What should I look for when monitoring unified memory performance on GB10?
Start from an idle state, then launch the model server and issue a query while observing system memory and GPU activity. You should see changes between idle and active states that align with zero-copy data sharing and hybrid AI inference described in the steps.
Next