Run Llama 3 on a Raspberry Pi 5 using ExecuTorch
Introduction
Set up the development environment
Set up ExecuTorch
Set up Llama 3
Run the model on a Raspberry Pi 5
Next Steps
Run Llama 3 on a Raspberry Pi 5 using ExecuTorch
Who is this for?
This is an introductory topic for anyone interested in running the Llama 3 model on a Raspberry Pi 5, and learning about techniques for running large language models (LLMs) in an embedded environment.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Use Docker to run Raspberry Pi OS on an Arm Linux server.
- Compile a Large Language Model (LLM) using ExecuTorch.
- Deploy the Llama 3 model on an edge device.
- Describe how to run Llama 3 on a Raspberry Pi 5 using ExecuTorch.
- Describe techniques for running large language models in an embedded environment.
Prerequisites
Before starting, you will need the following:
- An Arm Linux machine or an Arm cloud instance .
- A Raspberry Pi 5.
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
llama_main with a prompt, then try different prompts and command-line options. You can adjust the prompt and the number of CPU threads to explore the model’s behavior.