# Run a PyTorch LLM chatbot with KleidiAI on Arm

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/)
- [Run a Large Language model (LLM) chatbot on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama/)
- [Chatbot with Streamlit Frontend](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama-frontend/)
- [Next Steps](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/_next-steps/)

## About this Learning Path

|                |                       |
|----------------|-----------------------|
| Skill level:   | Introductory          |
| Reading time:  | 30 min                |
| Last updated:  | 12 Nov 2026           |
| Authors:       | Nikhil Gupta                      |
|                | Pareena Verma, Arm [GitHub](https://github.com/pareenaverma) [LinkedIn](https://linkedin.com/in/pareena-verma-7853607) |
|                | Nobel Chowdary Mandepudi, Arm    |
| Arm IP:        | [Neoverse](https://support.arm.com/?tab=compute-ip&Product%20Type=Infrastructure%20Processors) |
| Tags:          | [AI/ML](https://learn.arm.com/tag/ai-ml/)             |
|                | [AWS Graviton](https://learn.arm.com/tag/aws-graviton) |
|                | [Microsoft Azure Cobalt](https://learn.arm.com/tag/microsoft-azure-cobalt) |
|                | [Google Axion](https://learn.arm.com/tag/google-axion) |
|                | [Oracle Cloud Infrastructure (OCI) Ampere Compute](https://learn.arm.com/tag/oracle-cloud-infrastructure-oci-ampere-compute) |
|                | [Linux](https://learn.arm.com/tag/linux)               |
|                | [LLM](https://learn.arm.com/tag/llm)                   |
|                | [Generative AI](https://learn.arm.com/tag/generative-ai) |
|                | [Python](https://learn.arm.com/tag/python)             |
|                | [PyTorch](https://learn.arm.com/tag/pytorch)           |
|                | [Hugging Face](https://learn.arm.com/tag/hugging-face) |

### Who is this for?
This is an introductory topic for software developers interested in running LLMs using PyTorch on Arm-based servers.

### What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Download the Meta Llama 3.1 model from the Meta Hugging Face repository.
- 4-bit quantize the model using optimized INT4 KleidiAI Kernels for PyTorch.
- Run an LLM inference using PyTorch on an Arm-based CPU.
- Expose an LLM inference as a browser application with Streamlit as the frontend and Torchchat framework in PyTorch as the LLM backend server.
- Measure performance metrics of the LLM inference running on an Arm-based CPU.

### Prerequisites
Before starting, you'll need the following prerequisites:
- An [Arm-based instance](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/) with at least 16 CPUs from a cloud service provider or an on-premise Arm server.
