Run a local LLM chatbot on a Raspberry Pi 5
Introduction
Run a Large Language Model (LLM) chatbot on a Raspberry Pi 5
Next Steps
Run a local LLM chatbot on a Raspberry Pi 5
Review frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
The 8 GB RAM Raspberry Pi 5 model is preferred for exploring an LLM. Smaller memory configurations might restrict the model that you can run.
Check your system information to confirm the operating system and kernel report a 64-bit build. If not, reinstall Raspberry Pi OS using the 64-bit image.
Yes. The instructions apply to other Arm Linux systems with minor modifications where Raspberry Pi–specific steps appear.
Compare the model’s size to your available memory and leave headroom for the operating system and Python runtime. Select a model that fits comfortably within your RAM, then assess performance.
A Python script loads the chosen model and generates text responses in your terminal. If it starts and responds to prompts, your setup is working.