# Build a RAG pipeline on Arm-based NVIDIA DGX Spark

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/)
- [Explore building a RAG pipeline on Arm-based Grace–Blackwell systems](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/1_rag/)
- [Configure the RAG development environment and models](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/2_rag_setup/)
- [Add documents to the RAG vector database](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/2b_rag_setup/)
- [Build and run the RAG pipeline](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/3_rag_pipeline/)
- [Monitor unified memory performance](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/4_rag_memory_observation/)
- [Next Steps](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_rag/_next-steps/)

## About this Learning Path

| Skill level:    | Advanced         |
|------------------|------------------|
| Reading time:    | 1 hr             |
| Last updated:    | 28 Jul 2026      |

| Author:          | Odin Shen, Arm [GitHub](https://github.com/odincodeshen) [LinkedIn](https://linkedin.com/in/odin-shen-lmshen) |
|------------------|------------------|
| Arm IP:          | [Cortex-A](https://support.arm.com/?tab=compute-ip&Product%20Type=Application%20Processors) |
| Tags:            | ML, Linux, Python, llama.cpp, Hugging Face |

### Who is this for?
This is an advanced topic for developers who want to build a Retrieval-Augmented Generation (RAG) pipeline on the NVIDIA DGX Spark platform. You'll learn how Arm-based Grace CPUs handle document retrieval and orchestration, while Blackwell GPUs speed up large language model inference using the open-source llama.cpp REST server. This is a great fit if you're interested in combining Arm CPU management with GPU-accelerated AI workloads.

### What will you learn?
Upon completion of this Learning Path, you will be able to:
- Describe how a RAG system combines document retrieval and language model generation
- Deploy a hybrid CPU-GPU RAG pipeline on the GB10 platform using open-source tools
- Use the llama.cpp REST Server for GPU-accelerated inference with CPU-managed retrieval
- Build a reproducible RAG application that demonstrates efficient hybrid computing

### Prerequisites
Before starting, you will need the following:
- An NVIDIA DGX Spark system with at least 15 GB of available disk space

### Summary
You’ll build a Retrieval-Augmented Generation (RAG) pipeline on an Arm-based NVIDIA DGX Spark. You’ll prepare embedding and Llama 3.1 8B Instruct models, clean and chunk a sample corpus, create a FAISS index on the Grace CPU, and connect retrieval to GPU inference through the `llama.cpp` REST server. You’ll run queries and monitor unified memory during active RAG requests.

### Frequently asked questions

<details>
<summary>Do I need to complete the quantized LLM Learning Path before starting this one?</summary>
Complete the Learning Path [Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark](https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/) first. That background covers CPU and GPU builds of `llama.cpp` used in this RAG solution.
</details>

<details>
<summary>Which models should I use for embeddings and generation in this RAG pipeline?</summary>
Use `e5-base-v2` for embeddings and Llama 3.1 8B Instruct for generation. These models are prepared during the environment setup steps.
</details>

<details>
<summary>What should I expect after preparing documents and building the FAISS index?</summary>
The workflow converts your documents into clean, chunked text segments, then vectorizes and indexes them with FAISS. The result is a searchable vector database that returns the most relevant chunks for a query.
</details>

<details>
<summary>How do I know the retrieval and generation components are integrated correctly?</summary>
Run a query and confirm the response reflects information from your loaded documentation. Retrieval and indexing run on the Arm Grace CPUs, and generation uses the `llama.cpp` REST server on Blackwell GPUs.
</details>

<details>
<summary>What should I look for when monitoring unified memory performance on GB10?</summary>
Start from an idle state, then launch the model server and issue a query while observing system memory and GPU activity. You should see changes between idle and active states that align with zero-copy data sharing and hybrid AI inference described in the steps.
</details>