Who is this for?
This is an introductory topic for developers who are interested in running a Large Language Model (LLM) with rtp-llm on Arm-based servers.
What will you learn?
Upon completion of this learning path, you will be able to:
- Build rtp-llm on an Arm-based server.
- Download a Qwen model from Hugging Face.
- Run a Large Language Model with rtp-llm.
Prerequisites
Before starting, you will need the following:
- Any Arm Neoverse N2-based or Arm Neoverse V2-based instance running Ubuntu 22.04 LTS from a cloud service provider or an on-premise Arm server.
- For the server, at least four cores and 16GB of RAM, with disk storage configured up to at least 32 GB.