Nexmark is a benchmark suite specifically designed for evaluating and comparing the performance of streaming data processing systems, particularly those used in the context of online auction scenarios. It provides a set of representative queries and data generators that simulate real-world auction events.


  • Flink Standalone Cluster

  • JDK 1.8.x or higher (Nexmark scripts uses some tools of JDK)

  • ssh (sshd must be running to use the Flink and Nexmark scripts that manage remote components)

  • Install Maven:

                sudo apt install maven
  • Environment Variables - the following environment variable should be set on every node for the Flink and Nexmark scripts:

                # JAVA_HOME: points to the directory of your JDK installation.
    export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64
    # FLINK_HOME: points to the directory of your Flink installation.
    export FLINK_HOME=~/flink-benchmark/flink-1.17.2
  • Minimum requirements:

    • 3 worker nodes
    • 8 cores and 32 GB RAM on each machine
    • 800 GB SSD local disk

Nexmark Install and Config

  • Git clone the latest nexmark and build it on your master node:

                cd ~/flink-benchmark
    git clone https://github.com/nexmark/nexmark.git
    mv nexmark nexmark-src
    cd nexmark-src/nexmark-flink
    mv nexmark-flink.tgz ~/flink-benchmark
    cd ~/flink-benchmark
    tar xzf nexmark-flink.tgz
  • Copy the jars under nexmark/lib to flink/lib which contains the Nexmark source generator:

                cp ~/flink-benchmark/nexmark-flink/lib/*.jar ~/flink-benchmark/flink-1.17.2/lib
  • Nexmark and Flink Config:

    • Edit and replace the file nexmark-flink/conf/flink-conf.yaml:

                  #Set jobmanager.rpc.address: {JobManager_IP}
      #Set state.checkpoints.dir: file:///home/username/checkpoint (SSD Recommended)
      #Set state.backend.rocksdb.localdir: /home/username/rocksdb (SSD Recommended)
      #Set env.java.opts: [add -XX:+IgnoreUnrecognizedVMOptions]
      #Copy this file to [flink-1.17.2/conf/flink-conf.yaml]
      cp ~/flink-benchmark/nexmark-flink/conf/flink-conf.yaml ~/flink-benchmark/flink-1.17.2/conf
    • Replace flink-1.17.2/conf/sql-client-defaults.yaml with nexmark-flink/conf/sql-client-defaults.yaml:

                  cp ~/flink-benchmark/nexmark-flink/conf/sql-client-defaults.yaml ~/flink-benchmark/flink-1.17.2/conf
    • Edit nexmark-flink/conf/nexmark.yaml and set nexmark.metric.reporter.host:

                  #set nexmark.metric.reporter.host: {JobManager_IP}
    • scp Flink and Nexmark to your worker nodes:

                scp -r ~/flink-benchmark user@{TaskManager_IP}:~/