Orchestrate a persistent local AI agent with Hermes on NVIDIA DGX Spark
Introduction
Explore persistent AI runtime architecture on NVIDIA DGX Spark
Build the DGX Spark AI runtime foundation
Deploy Hermes Agent as an orchestration runtime
Add local LLM inference to Hermes Agent
Build persistent semantic memory for Hermes Agent
Add semantic retrieval and contextual reasoning to Hermes Agent
Add autonomous workspace cognition to Hermes Agent
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  1 hr 30 min
Last updated:  29 Jul 2026
Skill level:
  Advanced
Reading time:
  1 hr 30 min
Last updated:
  29 Jul 2026
Author:Odin Shen, Arm
Arm IP:Cortex-A
Tags: ML Linux Python Docker Ollama
Author:
  • Odin Shen, Arm
Arm IP:
Cortex-A
Tags:
ML Linux Python Docker Ollama

Who is this for?

This is an advanced topic for developers building persistent local AI agent systems on NVIDIA DGX Spark who want to use Arm Grace CPUs for orchestration and Blackwell GPUs for local LLM inference and embeddings.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Describe how persistent AI runtimes combine orchestration, semantic memory, and local inference
  • Build a continuously running local AI agent using Hermes Agent, Ollama, and Qdrant
  • Use Arm Grace CPUs to orchestrate event-driven AI workflows on NVIDIA DGX Spark
  • Deploy semantic memory and contextual retrieval pipelines using vector embeddings and Qdrant

Prerequisites

Before starting, you will need the following:

  • An NVIDIA DGX Spark system with at least 15 GB of available disk space
  • Familiarity with running Python scripts and basic Docker container workflows

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build a continuously running local AI agent on NVIDIA DGX Spark using Arm Grace CPUs for orchestration. You’ll configure Docker services with a persistent workspace, deploy Ollama, Qdrant, Open WebUI, and Hermes Agent. Then, you’ll connect Hermes to Ollama for document summaries and Qdrant embeddings. By the end, the services will monitor files, summarize documents, and provide contextual retrieval.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I know the base DGX Spark AI runtime is running correctly?
You’ll see containers for Ollama (inference), Qdrant (vector memory), and Open WebUI (browser access) running and healthy. Confirm that the persistent workspace exists and is mounted as expected.
Where should I put documents so Hermes picks them up automatically?
Place files in workspace/inbox/. Hermes watches that path and handles on_created() events to start the workflow.
I added Hermes but I don’t see summaries yet — what should I expect at this stage?
Not seeing summaries is expected. In its initial setup, Hermes acts as an orchestration and event layer, printing handling output but not invoking a language model until you connect it to Ollama.
After I connect Hermes to Ollama, what result should I expect to confirm inference is working?
When you add a new document to workspace/inbox/, Hermes sends the content to the local model and you see an AI-generated summary in the runtime output. This indicates the inference path is wired correctly.
How can I confirm persistent semantic memory is active in Qdrant?
Qdrant stores an embedding for each processed document. Check that new vector entries appear after Hermes handles files and are available for contextual retrieval in the workflow.
Next