Build a RAG application using Zilliz Cloud on Arm servers
Introduction
Overview and Install dependencies
Offline Data Loading
Launch the LLM Server
Online RAG
Next Steps
Build a RAG application using Zilliz Cloud on Arm servers
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Create a Dedicated cluster on AWS using Arm-based machines. You can also use self-hosted Milvus, although its setup is more involved.
After you select Create Cluster, check that the cluster appears in your Default Project with a running status. Continue after the cluster reports that it’s running.
No. You’ll run the llama.cpp server locally, so the OpenAI SDK can connect without a real API key.
Run the test code and confirm that it prints an embedding dimension and several floating-point values. The example reports a dimension of
384.Request access to Llama 3.1 through the Llama website. After you receive access, follow the instructions to host the model with llama.cpp on your Arm-based server.