Apache Kafka includes official performance testing utilities that allow you to measure throughput, latency, and end-to-end efficiency of your messaging system. These toolskafka-producer-perf-test.sh and kafka-consumer-perf-test.sh are bundled with Kafka’s standard installation and are designed for realistic performance evaluation of producers and consumers.
Before running the benchmarks, make sure your Kafka broker is already active in a separate terminal (as configured in the previous section). Now open two new terminal sessions; one for running the producer benchmark, and the other for the consumer benchmark.
The Producer Performance Test measures how quickly Kafka can publish messages to a topic and reports key performance metrics such as throughput, average latency, and percentile latencies.
Run the following command to simulate message production on your Azure Cobalt 100 Arm64 VM:
cd /opt/kafka
bin/kafka-producer-perf-test.sh \
--topic test-topic-kafka \
--num-records 1000000 \
--record-size 100 \
--throughput -1 \
--producer-props bootstrap.servers=localhost:9092
You should see output similar to:
1000000 records sent, 252589.0 records/sec (24.09 MB/sec), 850.85 ms avg latency, 1219.00 ms max latency, 851 ms 50th, 1184 ms 95th, 1210 ms 99th, 1218 ms 99.9th.
| Metric | Meaning |
|---|---|
| Records/sec | Number of messages successfully produced per second. Higher indicates better throughput. |
| MB/sec | Total data throughput in megabytes per second. |
| Avg latency | Average time (in milliseconds) for the producer to send a message and receive acknowledgment from the broker. |
| Max latency | The longest single message send time recorded. |
| 50th / 95th / 99th percentiles | Distribution of message send times. For example, 95% of messages completed under 1,184 ms in the sample output. |
The Consumer Performance Test measures how efficiently Kafka can read and process messages from a topic. It reports metrics such as total messages consumed, data throughput, and fetch rates, helping validate overall consumer-side performance on your Azure Cobalt 100 (Arm64) VM.
Run the following command in a new terminal:
cd /opt/kafka
bin/kafka-consumer-perf-test.sh \
--topic test-topic-kafka \
--bootstrap-server localhost:9092 \
--messages 1000000 \
--timeout 30000
You should see output similar to:
start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec, rebalance.time.ms, fetch.time.ms, fetch.MB.sec, fetch.nMsg.sec
2025-09-03 06:07:13:616, 2025-09-03 06:07:17:545, 95.3674, 24.2727, 1000001, 254517.9435, 3354, 575, 165.8564, 1739132.1739
Understanding the Metrics:
| Metric | Description |
|---|---|
data.consumed.in.MB | Total data consumed during the benchmark. |
MB.sec | Consumption throughput in megabytes per second. Higher values indicate better sustained read performance. |
data.consumed.in.nMsg | Total number of messages successfully consumed. |
nMsg.sec | Messages consumed per second (a key measure of consumer-side throughput). |
fetch.time.ms | Time spent retrieving messages from the broker. Lower values mean faster message delivery. |
fetch.nMsg.sec | Per-fetch message rate, useful for comparing network and I/O efficiency. |
rebalance.time.ms | Time spent coordinating consumer group assignments before actual consumption begins. |
The following results summarize Kafka producer and consumer benchmark performance on an Azure Cobalt 100 (Arm64) virtual machine, specifically a D4ps_v6 instance running Ubuntu Pro 24.04 LTS. These results validate Kafka’s stability and throughput consistency on Arm-based infrastructure.
| Metric | Value | Unit |
|---|---|---|
| Total Time Taken | 3.875 | Seconds |
| Data Consumed | 95.3674 | MB |
| Throughput (Data) | 24.6110 | MB/sec |
| Messages Consumed | 1,000,001 | Messages |
| Throughput (Messages) | 258,064.77 | Messages/sec |
| Rebalance Time | 3348 | Milliseconds |
| Fetch Time | 527 | Milliseconds |
| Fetch Throughput (Data) | 180.9629 | MB/sec |
| Fetch Throughput (Messages) | 1,897,535.10 | Messages/sec |
Interpretation: The consumer achieved over 258,000 messages per second, equivalent to ~24.6 MB/sec, with low fetch latency. A fetch throughput near 1.9 million messages/sec indicates efficient partition reads and network I/O handling on the Arm64 platform. Minimal rebalance and fetch times confirm Kafka’s responsiveness under sustained workloads.
| Metric | Records Sent | Records/sec | Throughput | Average Latency | Maximum Latency | 50th Percentile Latency | 95th Percentile Latency | 99th Percentile Latency | 99.9th Percentile Latency |
|---|---|---|---|---|---|---|---|---|---|
| Value | 1,000,000 | 257,532.8 | 24.56 | 816.19 | 1237.00 | 799 | 1168 | 1220 | 1231 |
| Unit | Records | Records/sec | MB/sec | ms | ms | ms | ms | ms | ms |
Interpretation: The producer sustained a throughput of ~257,500 records/sec (~24.5 MB/sec) with an average latency of 816 ms. The 95th percentile latency (1168 ms) and 99th percentile (1220 ms) show predictable network and I/O performance. Kafka maintained consistent throughput, even under full-speed production, with no message loss or broker errors reported.
When analyzing performance on Azure Cobalt 100 Arm64 virtual machines, you’ll notice that Kafka delivers stable and predictable results for both producers and consumers. The producer consistently achieves throughput between 23 MB/sec and 25 MB/sec, with average latencies below 900 ms. This means you can rely on efficient message delivery, even when handling high-volume workloads. On the consumer side, throughput remains strong at around 262,000 messages per second, and fetch performance scales nearly linearly, often exceeding 1.85 million messages per second internally. Throughout multiple benchmark runs, both producer and consumer tests demonstrate low jitter and consistent latency distribution, confirming that Kafka maintains reliable performance on Arm-based virtual machines.