Compare Neon and SVE with the Arm Performix Instruction Mix recipe

Verify vectorization with the Instruction Mix recipe

In the earlier build step, you created gpt2_neon and gpt2_sve. These binaries use the reference solutions in matmul_neon.cpp and matmul_sve.cpp, respectively.

Run the gpt2_neon binary with the following command to observe the speedup.

    

        
        
./build/gpt2_neon --model gpt2-medium "Once upon a time" -n 20

    

GPT-2 Neon runtime output on Arm Linux

The Neon implementation delivers a noticeable increase in token generation throughput. To evaluate the SVE implementation, run the same workload using the gpt2_sve binary instead of gpt2_neon. Performance gains will vary across systems, largely depending on the Scalable Vector Length (SVL) supported by the Arm Linux platform.

Rerun the Instruction Mix recipe with the gpt2_neon binary using the same recipe settings and workload arguments as baseline. After the run completes, select both runs and select Compare to open a comparison view.

The baseline profile is mostly scalar instructions. After you add Neon intrinsics, the instruction mix shifts toward Advanced SIMD (Neon) instructions, showing that the code is using Arm Neon hardware more effectively.

Neon versus scalar instruction mix

You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now using SVE hardware.

SVE versus baseline instruction mix

Compare throughput across kernels

Compare throughput across Neon and SVE.

Neon kernel

You can also inspect the Neon intrinsic implementation using Compiler Explorer, where the hot accumulation step (vacc) runs in ASIMD (Neon) registers such as v0:

SVE kernel

For variable-length vectorization, compare with an explicit SVE implementation that assumes SVE support, where the hot accumulation step (vacc) runs in SVE z registers with predicate-controlled loads and multiply-accumulate:

For a full-page view, open a Godbolt session with all three matmul kernels .

Measure speedup

Run the provided comparison script to measure tokens per second across all available binaries:

    

        
        ./compare_gpt2_variants.sh
Model: gpt2-medium
Prompt: Once upon a time
Tokens: 20
Runs: 1

== gpt2 ==
run 1: 3.04976 tok/s
avg: 3.049760 tok/s

== gpt2_neon ==
run 1: 11.3649 tok/s
avg: 11.364900 tok/s

== gpt2_sve ==
run 1: 13.907 tok/s
avg: 13.907000 tok/s

== gpt2_user ==
run 1: 3.04859 tok/s
avg: 3.048590 tok/s

        
    

These results show that intrinsics increase throughput from about 3 tok/s in the scalar baseline to about 13.9 tok/s with SVE.

What you’ve accomplished and what’s next

You’ve now verified vectorization using the Instruction Mix recipe, compared throughput across Neon and SVE, and measured tokens per second across the available binaries.

Next, you’ll use optimized libraries to push performance further.