Improve Python application performance using zlib-ng

Reduce time taken by a Python application to compress data

In the previous section, you learned how to build zlib-ng with Neon SIMD and ARMv8 CRC32 optimizations enabled.

In this section, you will accelerate the performance of an example Python application that compresses a large file and measure the performance difference when using zlib-ng.

Install necessary software packages

Ensure that python3 is available when you run python:

    

        
        
sudo apt install python-is-python3 -y

    

Create a large file to compress

Navigate to your home directory before creating the example files:

    

        
        
cd $HOME

    

To create an input file called largefile, use the dd command.

    

        
        
dd if=/dev/zero of=largefile count=1M bs=1024

    

Create an example Python file compression application

To create the Python application for compressing largefile, use a text editor to copy and save the following code in a file named zip.py.

    

        
        
import gzip

size = 16384

with open('largefile', 'rb') as f_in:
    with gzip.open('largefile.gz', 'wb') as f_out:
        while (data := f_in.read(size)):
            f_out.write(data)

f_out.close()

    

The Python file compression application will read largefile as input and write a compressed version as largefile.gz.

Compress the file using the Python application and default zlib

Run zip.py with the default zlib and time the execution.

    

        
        
time python zip.py

    

The output is similar to:

    

        
        real    0m4.662s
user    0m4.544s
sys     0m0.117s

        
    

Make a note of the real time.

Compress the file again using the Python application and zlib-ng

This time, use LD_PRELOAD to switch to zlib-ng and measure the performance difference.

Adjust the path to libz.so.1 as needed.

    

        
        
time LD_PRELOAD=/usr/local/lib/libz.so.1 python ./zip.py

    

The output is similar to:

    

        
        real    0m1.759s
user    0m1.654s
sys     0m0.105s

        
    

In this example, zlib-ng reduces compression time from 4.6 seconds to 1.8 seconds, roughly a 2.6x improvement — driven by the Neon-accelerated adler32 and inflate chunk copy routines.

What you’ve learned and what’s next

In this section, you used zlib-ng to accelerate the performance of an example Python file compression application. You compared the difference in performance between zlib and zlib-ng.

In the next section, you will learn how to use Linux perf to profile applications and look for zlib activity.