Benchmark Envoy on Google Cloud for Arm64 and x86_64 with Siege

How to run Envoy benchmarks with Siege on Arm64 in GCP

Siege is a lightweight HTTP load testing and benchmarking tool that simulates concurrent users making requests to a target service. It is useful for Envoy benchmarking because it measures availability, throughput, response time, and failure rates under load, thus helping evaluate Envoy’s performance as a proxy under real-world traffic conditions.

Follow the steps outlined to run Envoy benchmarks using Siege.

Install Siege (build from source)

Install required build tools:

    

        
        
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y wget make gcc

    

Download, extract and build Siege source:

    

        
        
wget http://download.joedog.org/siege/siege-4.1.6.tar.gz
tar -xvzf siege-4.1.6.tar.gz
cd siege-4.1.6
./configure
make
sudo make install

    

You have now successfully built and installed Siege on your Arm-based machine.

Verify installation:

    

        
        
siege --version

    

This checks if Siege is installed properly and shows the version number:

    

        
        SIEGE 4.1.6

Copyright (C) 2023 by Jeffrey Fulmer, et al.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.

        
    

Envoy benchmarking

To start, make sure Envoy is up and running with your config file (listening on port 10000):

    

        
        
envoy -c envoy_config.yaml --base-id 1

    

This runs the Envoy proxy with your configuration file (envoy_config.yaml) so it can start listening for requests.

On another terminal, verify that Envoy is running as expected with curl:

    

        
        
curl -v http://127.0.0.1:10000/get

    

Running from another terminal returns a 200 OK status, confirming that Envoy is running and successfully processing requests.

Run a time-based load test

There are different ways you can set up your benchmark tests. Here, you will run a benchmark for a fixed time instead of using a request count:

    

        
        
siege -c30 -t10S http://127.0.0.1:10000/get

    

This runs a load test where 30 users hit Envoy continuously for 10 seconds. After this, Siege will show performance results.

The output should show a list of HTTP requests and responses followed by a summary as shown:

    

        
        HTTP/1.1 200     0.03 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.03 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.03 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.02 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.42 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.03 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.03 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     1.17 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.81 secs:     383 bytes ==> GET  /get
HTTP/1.1 200     0.14 secs:     383 bytes ==> GET  /get

Lifting the server siege...
Transactions:                   1019 hits
Availability:                  99.80 %
Elapsed time:                  10.38 secs
Data transferred:               0.37 MB
Response time:                  0.29 secs
Transaction rate:              98.17 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   28.07
Successful transactions:        1019
Failed transactions:               2
Longest transaction:            2.89
Shortest transaction:           0.02

        
    

Understanding Envoy benchmark metrics and results with Siege

• Transactions: Total number of completed requests during the benchmark.
• Availability: Percentage of requests that returned a successful response.
• Elapsed Time: Total time taken to run the benchmark test.
• Data Transferred: Total amount of data exchanged during the test.
• Response Time: Average time taken for the server to respond to each request.
• Transaction Rate: Number of requests processed per second.
• Throughput: Volume of data transferred per second.
• Concurrency: Average number of simultaneous connections maintained.
• Successful Transactions: Total number of requests completed successfully.
• Failed Transactions: Total number of requests that failed.
• Longest Transaction: Maximum response time observed for a single request.
• Shortest Transaction: Minimum response time observed for a single request.

Benchmark summary on x86_64

To compare the benchmark results, the following were collected by running the same benchmark on a c3-standard-4 (4 vCPU, 16 GB memory) x86_64 VM in GCP, running RHEL 9:

Benchmark summary on Arm64

Results from the earlier run on the c4a-standard-4 (4 vCPU, 16 GB memory) Arm64 VM in GCP (RHEL 9):

Envoy performance benchmarking comparison on Arm64 and x86_64

When you compare the benchmarking performance results between the two instance types with the same vCPUs, you will notice that on the Google Axion C4A Arm-based instances:

• You have more successful transactions, fewer failures
• Lower response times, higher transaction rate, better throughput

You have successfully learned how to use Siege to benchmark Envoy on your Arm-based Axion Google cloud instance, validating both performance and reliability against similar x86_64 instances.