Accelerate LiteRT Models on Android with KleidiAI and SME2
Introduction
Explore LiteRT, XNNPACK, KleidiAI, and SME2
Create LiteRT models
Build the LiteRT benchmark tool
Benchmark the LiteRT model
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  45 min
Last updated:  19 Aug 2026
Skill level:
  Advanced
Reading time:
  45 min
Last updated:
  19 Aug 2026
Author:Jiaming Guo
Arm IP:Cortex-A Cortex-X Arm C1
Tags: ML Android C Python SME2
Author:
  • Jiaming Guo
Arm IP:
Cortex-A Cortex-X Arm C1
Tags:
ML Android C Python SME2

Who is this for?

This is an advanced topic for developers looking to leverage Arm's Scalable Matrix Extension 2 (SME2) instructions to accelerate LiteRT model inference on Android.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Understand how KleidiAI integrates with LiteRT
  • Build the LiteRT benchmark tool and enable XNNPACK and KleidiAI with SME2 support in LiteRT
  • Create LiteRT models that can be accelerated by SME2 through KleidiAI
  • Use the benchmark tool to evaluate and validate the SME2 acceleration performance of LiteRT models

Prerequisites

Before starting, you will need the following:

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll accelerate LiteRT inference on Android with KleidiAI SME2 microkernels through XNNPACK. First, you’ll create models using supported operators and data types, then build a KleidiAI-enabled benchmark and a baseline. After verifying SME2 support, you’ll run both on the same model and compare results, including fallback behavior for unsupported operators.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
What do I need on the Android device before running benchmarks?
Copy your LiteRT model file and two benchmark_model binaries to the device: one built with KleidiAI and SME2 enabled and one baseline build. Run both against the same model on the same device.
How do I check whether my Android device supports SME2?
From an adb shell, run cat /proc/cpuinfo and look for sme2 in the Features line. If it is present, the CPU supports SME2.
Which LiteRT operators are accelerated by SME2 through KleidiAI?
Only the subset of KleidiAI SME2 micro-kernels integrated into XNNPACK are accelerated. Operators outside the supported data types and quantization configurations use XNNPACK’s default implementation.
What result should I expect when comparing the two benchmark binaries?
The SME2-enabled binary demonstrates performance gains for models that use supported operators and data types. Both runs should complete successfully so you can compare their reported measurements.
What should I check if I don’t see an improvement with the SME2-enabled build?
Verify the device reports SME2 support, and confirm your model uses the supported operator configurations. Also ensure you ran the SME2-enabled binary under the same conditions as the baseline.
Next