Benchmark TinyStories with Google DDP and an AI agent
Ask for an Arm performance comparison
Use the connected phone to explore how Arm optimizations affect the inference performance of a small language model.
TinyStories is a language model consisting of short stories with a small vocabulary. It keeps the experiment small enough to build and run on the phone inside Termux, an Android terminal environment. KleidiAI provides optimized Arm CPU kernels that llama.cpp can use for inference.
Continue in the same agy Developer Device Platform (DDP) session and enter the second prompt:
On our DDP phone, compare a small TinyStories model's CPU inference
with and without Arm KleidiAI optimizations using the latest stable
Termux and llama.cpp releases. Check output agreement and which Arm
kernels actually run, including SVE2 where usable. Keep the model,
quantization, inputs, thread count, and CPU placement the same for both
paths. Repeat with the same model and settings, and report prompt-processing and
token-generation uplift. Save the results locally and release the
phone within our approved limits.
The prompt gives Antigravity the workload, comparison, and expected result. Antigravity can choose compatible software and model files, prepare the phone, and collect the measurements. You don’t need to attach these pages or copy version numbers and build settings into the conversation.
Let Antigravity prepare and run the experiment
Antigravity should obtain the latest stable releases from the official Termux releases and llama.cpp releases . After obtaining the stable releases, Antigravity installs the tools and builds and runs the workload on the phone. It records the versions that it selected in the saved report so that you can revisit the experiment later.
The Arm feature check is part of that work. Antigravity needs to establish the following details:
- Which optimizations the phone exposes
- Which optimizations the selected model and runtime can use
- Which kernels run during inference
If Scalable Vector Extension (SVE) or SVE2 isn’t usable for that combination, Antigravity should generate an explanation for that. It should identify the compatible path that it measured.
For a fair comparison, Antigravity should keep GPU offload disabled and the following settings consistent:
- Model
- Quantization
- Inputs
- Thread count
- CPU placement
It should check output agreement before timing and use repeated, warmed-up measurements. You can review the resulting comparison without choosing compiler flags, kernel dispatch settings, or numerical tolerances yourself.
Review the performance report
Look for separate results for prompt processing and token generation. The report should show the baseline and optimized throughput, the uplift, and the variation across measurements. Check the explanation generated by Antigravity of the kernels responsible for the result, including any fallback or regression.
The following reference results were measured on November 5, 2026. The measurements were made on a physical Pixel 11 with Tensor G6 and Android 17 (API 37). Each value is the median of five alternating measurement pairs, using the same TinyStories Q4_0 model and one CPU thread:
| Workload | CPU baseline, tokens/s | KleidiAI, tokens/s | Speedup | Throughput uplift |
|---|---|---|---|---|
| Prompt processing, 64 tokens | 4,833.49 | 9,925.77 | 2.05× | +105.4% |
| Token generation, 32 tokens | 1,029.68 | 1,375.95 | 1.34× | +33.6% |
Higher tokens per second is better. Calculate speedup = optimized / baseline and throughput uplift (%) = 100 × (speedup − 1) from the median throughput for each path. These results measure warmed-up inference and exclude model loading and initial weight packing.
Antigravity selects the latest stable releases and a compatible model for your run. The results that you see might differ from these reference values.
Confirm the results are saved and the phone is released
The prompt includes saving the results and releasing the phone. Check that Antigravity gives you a readable local report with the measurements, device details, software versions, and kernel evidence.
Antigravity should cancel the recorded DDP session and confirm that its cloud state is FINISHED or EXPIRED. It should then close the viewer and connection. Closing a window alone doesn’t stop device-time billing. If the task stops early, ask Antigravity to release the DDP session and verify that it has ended.
What you’ve accomplished
You’ve used Antigravity CLI and natural language to connect to remote Arm hardware and investigate a performance question. Antigravity handled device access, software selection, and setup. It also handled kernel checks, measurement, and cleanup. You chose the task and spending limits, then reviewed the screen, results, and explanation.
You can adapt this Learning Path to test your own workloads on remote Arm hardware.