Benchmark Gemma 4 CPU inference with KleidiAI and SME2
Introduction
Set up the Gemma 4 benchmark workspace
Install prerequisites on macOS and download Gemma 4
Build and compare benchmarks on macOS
Next Steps
Benchmark Gemma 4 CPU inference with KleidiAI and SME2
Who is this for?
Software developers and performance engineers evaluating Gemma 4 CPU inference with LiteRT-LM, KleidiAI, and XNNPACK on SME2 hardware.
What will you learn?
After completing this Learning Path, you'll be able to perform the following tasks:
- Build LiteRT-LM with baseline and optimized XNNPACK kernels.
- Download and verify the Gemma 4 E2B LiteRT-LM model.
- Measure prefill and decode throughput at one and four CPU threads.
- Compare throughput and verify the accelerated operator.
Prerequisites
Before starting, you'll need the following prerequisites:
- An SME2-capable device , such as an M4-family Mac
- Git, Homebrew, and Xcode Command Line Tools
- At least 25 GB of free disk space for model files and local builds
Summary
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You’ll benchmark Gemma 4 CPU inference on an SME2-capable Mac with LiteRT-LM and KleidiAI. First, you’ll confirm SME2 support and prepare baseline and optimized XNNPACK sources. Next, you’ll install the build tools and verify the downloaded model. Then, you’ll run three-iteration benchmarks at one and four CPU threads and compare prefill and decode throughput. Finally, you’ll check the model response and use a separate profiled run to verify the accelerated operator.