Run a local AI agent with Ollama to visualize CPU orchestration on Arm
Introduction
Understand the agent and how work is split between CPU and GPU
Set up your environment before running the agent
Serve a model locally with Ollama
Understand the concierge agent code
Run the agent and read the timeline
Understand the CPU's role in the agentic workflow
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  45 min
Last updated:  13 Jul 2026
Skill level:
  Introductory
Reading time:
  45 min
Last updated:
  13 Jul 2026
Author:Jaidev Singh Chadha, Arm
Arm IP:Cortex-A Neoverse
Tags: ML Linux macOS Python Ollama Generative AI LLM
Author:
  • Jaidev Singh Chadha, Arm
Arm IP:
Cortex-A Neoverse
Tags:
ML Linux macOS Python Ollama Generative AI LLM

Who is this for?

This is an introductory topic for developers who want to build a local, privacy-friendly AI agent on Arm hardware and visualize how the CPU orchestrates an agentic workflow around a locally served Large Language Model (LLM).

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Set up a Python environment and obtain a Serper web search API key for the agent
  • Serve an LLM locally with Ollama and select a model that fits your hardware
  • Understand how the CPU orchestrates an agentic workflow while the GPU handles model inference
  • Run the agent and interpret the CPU and GPU timeline it produces for each query

Prerequisites

Before starting, you will need the following:

  • An Arm-based computer running Linux or macOS, such as an Apple silicon MacBook or an NVIDIA DGX Spark
  • Familiarity with running Python scripts from the terminal

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll run a local concierge agent on an Arm machine and examine how the workload splits between CPU orchestration and GPU-backed model inference. You’ll set up a Python environment, configure a Serper web search API key, and serve Gemma 3 locally with Ollama. As the agent answers your research-style questions, you’ll use its per-query timeline to see how much of web search, scraping, ranking, deduplication, and extraction runs on the CPU.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I know Ollama is ready before running the agent?
Confirm Ollama is running and serving the gemma3:4b model. The local API at http://localhost:11434 should be available before starting the script.
What do I need in the terminal environment when I start the script?
Activate your Python virtual environment and set SERPER_API_KEY in the same terminal session. The agent reads this variable at runtime for web search.
What result should I expect when the agent starts correctly?
You should see a greeting that explains the agent browses websites in real time and a prompt asking for your question. Type quit or exit to end the session.
How do I read the CPU and GPU timeline the agent prints?
CPU entries in the timeline correspond to tools (web search, scraping) and the orchestration pipeline (query expansion, ranking, deduplication, extraction). GPU entries correspond to the Gemma model running through Ollama. The agentic chain interleaves these steps.
Where can I see how the workflow is organized in the code?
Open concierge_agent.py. It groups the tools, the LLM “brain,” the orchestration pipeline, and the agentic chain, and indicates which components run on the CPU or GPU.
Next