Who is this for?
This is an advanced topic for software developers who want to learn how to use Kafka and Zookeeper.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Install Zookeeper and Kafka.
- Configure Zookeeper to work with Kafka.
- Test write and read events into the Kafka cluster.
- Deploy a cluster automatically on AWS.
- Deploy a cluster automatically on GCP.
Prerequisites
Before starting, you will need the following:
- Seven physical Arm machines or cloud instances with either Ubuntu or Debian installed
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You’ll deploy a three-node Apache Kafka cluster on Arm, beginning with a three-node ZooKeeper ensemble. First, you’ll prepare network ports and configure the brokers. Then, you’ll start the services in order and validate end-to-end messaging by creating a topic, producing events, and reading them from a client. You’ll also use Terraform and Ansible to automate the same topology on AWS Graviton.
Frequently asked questions
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
The cluster is ready when all three ZooKeeper nodes are started and reachable from the Kafka nodes. The Kafka configuration will reference the three ZooKeeper IPs, so confirm those addresses are correct and accessible.
Replace
zk_1_ip, zk_2_ip, and zk_3_ip with the IP addresses of the three ZooKeeper nodes that you set up. Use the same addresses that you used when bringing up the ZooKeeper cluster.Open ports
8080, 2888, 3888, 2181, and 9092 in the security groups for these machines. Ensure that the rules allow traffic between cluster nodes and from the client where you run Kafka commands.You should see the consumer read the same message that you wrote in the producer terminal. This verifies that the Kafka cluster is working.
Run the automation from any computer with the required tools installed. The AWS deployment uses Terraform and Ansible to provision a three-node ZooKeeper cluster, a three-node Kafka cluster, and one client on AWS Graviton.