Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Complete the Learning Path
Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark
first. That background covers CPU and GPU builds of
llama.cpp used in this RAG solution.Use
e5-base-v2 for embeddings and Llama 3.1 8B Instruct for generation. These models are prepared during the environment setup steps.The workflow converts your documents into clean, chunked text segments, then vectorizes and indexes them with FAISS. The result is a searchable vector database that returns the most relevant chunks for a query.
Run a query and confirm the response reflects information from your loaded documentation. Retrieval and indexing run on the Arm Grace CPUs, and generation uses the
llama.cpp REST server on Blackwell GPUs.Start from an idle state, then launch the model server and issue a query while observing system memory and GPU activity. You should see changes between idle and active states that align with zero-copy data sharing and hybrid AI inference described in the steps.