Run and benchmark BitNet-2B inference on Arm CPUs with Litespark-Inference
Introduction
Understand Litespark-Inference and BitNet b1.58
Run BitNet-2B with Litespark-Inference
Benchmark Litespark-Inference on Arm
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Introductory
Reading time:  25 min
Last updated:  03 Aug 2026
Skill level:
  Introductory
Reading time:
  25 min
Last updated:
  03 Aug 2026
Authors:Nii Osae Osae Dade, Mindbeam
Tony Morri, Mindbeam
Sayandip Pal, Mindbeam
Arm IP:Neoverse Cortex-A
Tags: ML Linux macOS Python
Authors:
  • Nii Osae Osae Dade, Mindbeam
  • Tony Morri, Mindbeam
  • Sayandip Pal, Mindbeam
Arm IP:
Neoverse Cortex-A
Tags:
ML Linux macOS Python

Who is this for?

This is an introductory topic for developers who want to run a large language model (LLM) on a CPU without a GPU using Litespark-Inference.

What will you learn?

After completing this Learning Path, you'll be able to perform the following tasks:

  • Run BitNet-2B from the Litespark-Inference CLI or from Python using the high-level BitNet API.
  • Pick the right embed dtype (BF16, INT8, or INT4) for the memory versus quality trade-off you want.
  • Compare Litespark-Inference with PyTorch Transformers for memory, time to first token, and throughput.
  • Measure energy per token on Apple silicon.

Prerequisites

Before starting, you'll need the following prerequisites:

  • An Arm Linux machine or macOS machine with Apple Silicon, running Python 3.10 or later. The machine can be anything from a Raspberry Pi 5 to an Arm server.
  • Litespark-Inference installed by following the instructions in the Litespark-Inference install guide .
  • About 5 GB of free disk for the BitNet-2B model that downloads on first run.

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll run BitNet-2B on Arm CPUs with Litespark-Inference from the command line and Python. You’ll verify the Arm Neon kernel and compare BF16 and INT4 embedding memory use. Then, you’ll benchmark the torchless runtime against PyTorch for memory, time to first token, and throughput. On Apple silicon, you’ll also measure energy per token.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I limit the number of tokens generated from the CLI?
You can add --max-tokens to litespark-inference generate. For example, use --max-tokens 64 to limit the response to 64 new tokens.
Which embedding data type should I start with, and how do I set it?
Start with BF16 as the reference by adding --embed-dtype bf16. You can use INT8 or INT4 to reduce memory use, but validate output quality for your workload.
How do I verify that Litespark-Inference is using the Arm-optimized torchless runtime?
Run python -m litespark_inference.torchless info. On Arm, confirm that the kernel name contains neon and that OpenMP is True. The generation command also identifies the torchless runtime at startup.
How do I compare Litespark-Inference with a PyTorch baseline?
Add --pytorch to litespark-benchmark. The command runs the torchless and PyTorch passes, then reports memory use, time to first token, prompt prefill throughput, and token-generation throughput for comparison.
How do I measure energy per token on Apple silicon?
Refresh your sudo credentials with sudo -v, then add --power to litespark-benchmark. If the results report available: false, your system doesn’t expose a supported energy counter.
Next