# Build a RAG pipeline on Arm-based NVIDIA DGX Spark

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/)
- [Explore building a RAG pipeline on Arm-based Grace–Blackwell systems](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/1_rag/)
- [Configure the RAG development environment and models](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/2_rag_setup/)
- [Add documents to the RAG vector database](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/2b_rag_setup/)
- [Build and run the RAG pipeline](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/3_rag_pipeline/)
- [Monitor unified memory performance](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/4_rag_memory_observation/)
- [Next Steps](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/_next-steps/)

## About this Learning Path

| Skill level:         | Advanced           |
|----------------------|--------------------|
| Reading time:        | 1 hr               |
| Last updated:        | 28 Jul 2026        |
| Author:              | Odin Shen, Arm     |
| Arm IP:              | [Cortex-A](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors) |
| Tags:                | ML, Linux, Python, llama.cpp, Hugging Face |

### Who is this for?
This is an advanced topic for developers who want to build a Retrieval-Augmented Generation (RAG) pipeline on the NVIDIA DGX Spark platform. You'll learn how Arm-based Grace CPUs handle document retrieval and orchestration, while Blackwell GPUs speed up large language model inference using the open-source llama.cpp REST server. This is a great fit if you're interested in combining Arm CPU management with GPU-accelerated AI workloads.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Describe how a RAG system combines document retrieval and language model generation
- Deploy a hybrid CPU-GPU RAG pipeline on the GB10 platform using open-source tools
- Use the `llama.cpp` REST Server for GPU-accelerated inference with CPU-managed retrieval
- Build a reproducible RAG application that demonstrates efficient hybrid computing

### Prerequisites
Before starting, you'll need the following prerequisites:
- An NVIDIA DGX Spark system with at least 15 GB of available disk space

### Summary
You’ll build a Retrieval-Augmented Generation (RAG) pipeline on an Arm-based NVIDIA DGX Spark. You’ll prepare embedding and Llama 3.1 8B Instruct models, clean and chunk a sample corpus, create a FAISS index on the Grace CPU, and connect retrieval to GPU inference through the `llama.cpp` REST server. You’ll run queries and monitor unified memory during active RAG requests.

### Frequently asked questions

**Do I need to complete the quantized LLM Learning Path before starting this one?**

Complete the Learning Path [Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/) first. That background covers CPU and GPU builds of `llama.cpp` used in this RAG solution.

**Which models should I use for embeddings and generation in this RAG pipeline?**

Use `e5-base-v2` for embeddings and Llama 3.1 8B Instruct for generation. These models are prepared during the environment setup steps.

**What should I expect after preparing documents and building the FAISS index?**

The workflow converts your documents into clean, chunked text segments, then vectorizes and indexes them with FAISS. The result is a searchable vector database that returns the most relevant chunks for a query.

**How do I know the retrieval and generation components are integrated correctly?**

Run a query and confirm the response reflects information from your loaded documentation. Retrieval and indexing run on the Arm Grace CPUs, and generation uses the `llama.cpp` REST server on Blackwell GPUs.

**What should I look for when monitoring unified memory performance on GB10?**

Start from an idle state, then launch the model server and issue a query while observing system memory and GPU activity. You should see changes between idle and active states that align with zero-copy data sharing and hybrid AI inference described in the steps.