Run a local LLM chatbot on a Raspberry Pi 5
Introduction
Run a Large Language Model (LLM) chatbot on a Raspberry Pi 5
Review
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  30 min
Last updated:  16 May 2024
Skill level:
  Introductory
Reading time:
  30 min
Last updated:
  16 May 2024
Author:Jason Andrews, Arm
Arm IP:Cortex-A
Tags: ML Linux LLM GenAI Raspberry Pi Python
Author:
Jason Andrews, Arm
Arm IP:
Cortex-A
Tags:
ML Linux LLM GenAI Raspberry Pi Python

Who is this for?

This is an introductory topic for anyone interested in running a local Large Language Model on a Raspberry Pi 5.

What will you learn?

Upon completion of this learning path, you will be able to:

  • Install the Python version of llama.cpp on your Raspberry Pi 5.
  • Download an LLM from Hugging Face.
  • Assess LLM memory size and performance.
  • Run the LLM on your Raspberry Pi 5 using Python bindings for llama.cpp.

Prerequisites

Before starting, you will need the following:

  • A Raspberry Pi 5 running Raspberry Pi OS.
Next