View the GPT-2 benchmark results

At this point, the results directory should contain three GPT-2 result files:

• gpt2_pytorch.json
• gpt2_portable.json
• gpt2_xnnpack.json

Use this Python script to display results in a browser dashboard. Serve the result dashboard from the Raspberry Pi:

    

        
        
course_support/serve_results.py \
  --results ~/executorch_on_arm/module-1-transformer-inference-with-xnnpack/results \
  --host 127.0.0.1 \
  --port 8080

    

Then open the dashboard locally on your development machine:

    

        
        
http://localhost:8080

    

The dashboard should contain the three GPT-2 results generated so far.

The reference results below show the expected pattern, although your timings will likely differ:

In the reference run, XNNPACK reduced mean latency from 604.44 ms to 383.09 ms compared with PyTorch eager execution. This is approximately a 1.6× speedup, or a 37% reduction in mean latency. The median values show a similar result: 603.07 ms for PyTorch eager and 370.46 ms for XNNPACK. The single run of the portable kernel artifact took around 63.4 seconds. It should be immediately clear that while the portable kernels provided functional execution on-device, they did so without performance optimizations.

Despite not all operators being delegated and delegation being fragmented, the XNNPACK path achieved lower latency than PyTorch eager, and significantly lower latency than the portable kernel artifact. This indicates that the delegated regions included computationally significant transformer operations, optimizing bottleneck areas.

The XNNPACK .pte artifact was 154.45 MB, or approximately 31%, larger than the portable artifact. Whether this increase is acceptable depends on the target’s storage and packaging constraints.