# Run a Large Language Model (LLM) chatbot with PyTorch using KleidiAI on Arm servers

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/)
- [Run a Large Language model (LLM) chatbot on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama/)
- [Chatbot with Streamlit Frontend](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama-frontend/)
- [Next Steps](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/_next-steps/)

## About this Learning Path

|  |  |
| --- | --- |
| Skill level: | Introductory |
| Reading time: | 30 min |
| Last updated: | 09 Sep 2026 |

### Authors
- Nikhil Gupta
- Pareena Verma, Arm  
  [GitHub](https://github.com/pareenaverma)  
  [LinkedIn](https://linkedin.com/in/pareena-verma-7853607)
- Nobel Chowdary Mandepudi, Arm

### Arm IP
[Neoverse](https://support.arm.com/?tab=compute-ip&Product%20Type=Infrastructure%20Processors)

### Tags
- [ML](https://learn.arm.com/tag/ml)
- [AWS Graviton](https://learn.arm.com/tag/aws-graviton)
- [Microsoft Azure Cobalt](https://learn.arm.com/tag/microsoft-azure-cobalt)
- [Google Axion](https://learn.arm.com/tag/google-axion)
- [Oracle Cloud Infrastructure (OCI) Ampere Compute](https://learn.arm.com/tag/oracle-cloud-infrastructure-oci-ampere-compute)
- [Linux](https://learn.arm.com/tag/linux)
- [LLM](https://learn.arm.com/tag/llm)
- [Generative AI](https://learn.arm.com/tag/generative-ai)
- [Python](https://learn.arm.com/tag/python)
- [PyTorch](https://learn.arm.com/tag/pytorch)
- [Hugging Face](https://learn.arm.com/tag/hugging-face)

### Who is this for?
This is an introductory topic for software developers interested in running LLMs using PyTorch on Arm-based servers.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Download the Meta Llama 3.1 model from the Meta Hugging Face repository.
- 4-bit quantize the model using optimized INT4 KleidiAI Kernels for PyTorch.
- Run an LLM inference using PyTorch on an Arm-based CPU.
- Expose an LLM inference as a browser application with Streamlit as the frontend and Torchchat framework in PyTorch as the LLM backend server.
- Measure performance metrics of the LLM inference running on an Arm-based CPU.

### Prerequisites
Before starting, you'll need the following prerequisites:
- An [Arm-based instance](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/) with at least 16 CPUs from a cloud service provider or an on-premise Arm server.