Profile GPT-2 inference with the Arm Performix Instruction Mix recipe
Introduction
Understand profiling with Arm Performix Instruction Mix
Set up and run a GPT-2 baseline
Find GPT-2 hotspots and profile with the Arm Performix Instruction Mix recipe
(Optional) Optimize matmul with vector intrinsics
Compare Neon and SVE with the Arm Performix Instruction Mix recipe
Accelerate execution with KleidiAI
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  45 min
Last updated:  15 Jul 2026
Skill level:
  Introductory
Reading time:
  45 min
Last updated:
  15 Jul 2026
Authors:Kieran Hejmadi, Arm
Oliver Grainge
Arm IP:Neoverse
Tags: Performance and Architecture Linux Arm Performix C LLM Neon SVE
Authors:
  • Kieran Hejmadi, Arm
  • Oliver Grainge
Arm IP:
Neoverse
Tags:
Performance and Architecture Linux Arm Performix C LLM Neon SVE

Who is this for?

This is an introductory topic for developers who want to get started using the Arm Performix Instruction Mix recipe through a practical example.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Use the Instruction Mix recipe to combine static disassembly with runtime sampling to show execution behavior
  • Build and run the GPT-2 inference example on an Arm Linux server
  • Identify why matrix multiplication dominates runtime and how vectorization changes the instruction mix
  • Compare throughput and instruction mix across scalar, Neon, SVE, and KleidiAI implementations

Prerequisites

Before starting, you will need the following:

  • Access to Arm Performix configured with a remote Arm Linux target. For setup, see the Arm Performix install guide .
  • Basic understanding of C++ and compiler optimization
  • Basic understanding of matrix multiplication
  • Basic understanding of writing SIMD code with Neon or SVE

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll build a GPT-2 inference example on an Arm Linux target and use Arm Performix to analyze execution behavior. After running the Code Hotspots recipe to locate the dominant matmul kernel, you’ll use the Instruction Mix recipe to compare scalar and vector execution patterns on Arm Neoverse. You’ll then compare the scalar gpt2 baseline with gpt2_neon, gpt2_sve, optional gpt2_user, and the KleidiAI-backed gpt2_kai_sve path, validating vectorization and throughput changes with Performix data and tokens-per-second results.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
Which Performix recipe should I run first to find the bottleneck?
Start with the Code Hotspots recipe. Set the launch command to the baseline binary and use -n 150 to bias the profile toward inference time, then review the top functions.
How do I know that matrix multiplication is the hotspot before I change any code?
The Code Hotspots report ranks functions by sampled time, so kernels::matmul_ref() should appear near the top. Use that evidence to decide where to focus subsequent Instruction Mix profiling.
Where do I add my Neon or SVE intrinsics, and what part of the loop should I change?
Edit src/kernels/matmul_user.cpp. Focus on the accumulation loop, acc += row[j] * x[j];, and consider lane utilization, loop unrolling, and how you handle the tail.
What binaries should I expect after building, and where are the reference implementations?
The build produces the scalar baseline gpt2, reference variants gpt2_neon and gpt2_sve, and gpt2_user when the user kernel is enabled. The reference kernels are implemented in matmul_neon.cpp and matmul_sve.cpp.
How do I verify that my vectorization worked using Performix?
Profile your rebuilt binary with the Instruction Mix recipe and compare the proportion of vector instructions with the baseline. You can also run the reference gpt2_neon and gpt2_sve and compare instruction mix and the program’s tokens-per-second summary.
Next