Explore SME matrix operations with interactive visualizations
Introduction
Explore the ZA tile layout
Transpose a 32-bit matrix in place with ZA
Transpose a matrix with an incomplete edge
Multiply matrices with vector outer products
Transpose and pack matrix A
Pad matrix B to the ZA tile width
Trace packed GEMM through ZA
Compare a GEMM with gather loads
Interleave matrix A
Trace GEMM with interleaved matrix A
Next Steps
Explore SME matrix operations with interactive visualizations
Introduction
Explore the ZA tile layout
Transpose a 32-bit matrix in place with ZA
Transpose a matrix with an incomplete edge
Multiply matrices with vector outer products
Transpose and pack matrix A
Pad matrix B to the ZA tile width
Trace packed GEMM through ZA
Compare a GEMM with gather loads
Interleave matrix A
Trace GEMM with interleaved matrix A
Next Steps
Understand gather loads
The previous General Matrix Multiplication (GEMM) began with an input matrix A that was already packed and transposed. It isn’t always practical or desirable to transform the input data. The Scalable Vector Extension (SVE) supports Gather Load and Scatter Store. With these instructions, contiguous data in Z registers can be gathered from or scattered to non-contiguous addresses in memory.
These instructions are available in Streaming SVE mode only if FEAT_SME_FA64 is implemented and enabled. Scalable Matrix Extension (SME) functionality requires Streaming SVE mode to be enabled. If FEAT_SME_FA64 isn’t enabled, Gather Load and Scatter Store are illegal instructions while in Streaming SVE mode.
The following visualization uses Gather Loads to access columns of matrix A data. The elements of each column are separated in memory by the length of a full row of matrix A.
Gather Loads and Scatter Stores stress the memory system more than contiguous loads. They might potentially require multiple cache line loads before the operation can complete. Both these effects can impact performance.
Compare the input path
To keep this Learning Path open while exploring, open the gather-load GEMM visualization in a separate browser tab. Move the tab to a separate window to view the visualization and Learning Path side by side.
Focus on how matrix A data is loaded into Z2 and Z3:
| Stage | Number of steps | Operation |
|---|---|---|
| Load the first inputs | 4 | Read the first row of B Input Data into Z0–Z1 and the first column of A Input Data into Z2–Z3. |
| Calculate the first outer products | 4 | Use Z0–Z3 to calculate the first matrix outer products. |
| Process the second inputs | 8 | Read the second row of B and second column of A. Calculate the matrix outer products with accumulation of the result. |
| Process the third inputs | 8 | Repeat for the third row of B and third column of A. |
| Process the fourth inputs | 8 | Repeat for the fourth row of B and fourth column of A. |
| Store the first results | 8 | Write ZA0 and ZA1 to Output Matrix Data. |
| Store the remaining results | 8 | Write ZA2 and ZA3 to Output Matrix Data. |
The B Input Data loads use contiguous addresses. The four 32-bit words loaded from A Input Data come from separate rows, so their addresses are 16 bytes apart.
What you’ve learned and what’s next
You’ve learned that input data can be loaded from different memory layouts. The output of the matrix operation is the same. However, the memory layout of the input data can have a performance impact.
Next, you’ll view a visualization that rearranges matrix A in memory. The rearrangement means that data is loaded from contiguous addresses.