# Identify Arm CPU topology, cache hierarchy, and NUMA configuration

## In this learning path

- [Introduction](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/)
- [Identify Arm CPU topology, cache hierarchy, and NUMA configuration](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/system-overview/)
- [Analyze Arm cache hierarchy and performance characteristics](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/cache-hierarchy/)
- [Measure Arm cache and memory latency using ASCT pointer chase](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/pointer-chase-latency/)
- [Measure Arm single-core memory bandwidth with ASCT](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/streaming-bandwidth/)
- [Measure Arm multi-core memory bandwidth and loaded latency with ASCT](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/multicore-bandwidth/)
- [Compare Arm memory subsystem performance across systems](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/comparative-analysis/)
- [Next Steps](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/_next-steps/)

## Memory subsystem impact on Arm server performance
Memory behavior is one of the primary determinants of compute performance for memory-bound applications. The CPU-side memory subsystem, including caches, interconnects, and DRAM, directly impact performance. Understanding latency, streaming bandwidth, and how caches and cores share resources is essential for diagnosing bottlenecks and tuning software on Arm servers.

## Identify CPU, cache, and NUMA topology
Before you can characterize a memory subsystem, you need to understand how the system is organized. The number of cores, how they are grouped into clusters, the DRAM type, and the operating system all influence memory behavior. Knowing this topology helps interpret benchmark results.

The examples use two AWS Graviton instances:
- AWS Graviton2, `c6g.16xlarge`, instance with Arm Neoverse N1 cores.
- AWS Graviton4, `c8g.16xlarge`, instance with Arm Neoverse V2 cores.

Both systems run Ubuntu 24.04 with 64 CPUs, 128 GB RAM, and 32 GB of storage.

These are only examples, and you can follow the same steps on any Arm Linux system.

If you want to follow along with the same instances, create two EC2 instances using the AWS console or CLI:
- A `c6g.16xlarge` instance running Ubuntu 24.04 (Graviton2)
- A `c8g.16xlarge` instance running Ubuntu 24.04 (Graviton4)

Make sure both instances are accessible via SSH before continuing.

## Collect basic system information
Start by setting a descriptive hostname on each system. This makes it easier to identify results later when you use `$(hostname)` in commands.

On the Graviton4 instance, run:
```bash
sudo hostnamectl set-hostname graviton4-c8g
```

On the Graviton2 instance, run:
```bash
sudo hostnamectl set-hostname graviton2-c6g
```

Log out and log back in for the hostname to appear in your shell prompt.

Record the kernel version and operating system. Run the following command on your system:
```bash
uname -a && cat /etc/os-release
```

Both systems have the same software, but this is example output, the kernel version may differ depending on updates:
```
__output__ Linux graviton2-c6g 6.17.0-1007-aws #7~24.04.1-Ubuntu SMP Thu Jan 22 20:37:30 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux
__output__ PRETTY_NAME="Ubuntu 24.04.4 LTS"
__output__ NAME="Ubuntu"
__output__ VERSION_ID="24.04"
__output__ VERSION="24.04.4 LTS (Noble Numbat)"
__output__ VERSION_CODENAME=noble
__output__ ID=ubuntu
__output__ ID_LIKE=debian
__output__ HOME_URL="https://www.ubuntu.com/"
__output__ SUPPORT_URL="https://help.ubuntu.com/"
__output__ BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
__output__ PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
__output__ UBUNTU_CODENAME=noble
__output__ LOGO=ubuntu-logo
```

Print the CPU information:
```bash
lscpu
```

Pay close attention to the `lscpu` output. It gives you the CPU model, the number of cores, threads per core, sockets, and NUMA node configuration. Here is an example from the Graviton2 instance:
```
__output__ Architecture:                aarch64
__output__   CPU op-mode(s):            32-bit, 64-bit
__output__   Byte Order:                Little Endian
__output__ CPU(s):                      64
__output__   On-line CPU(s) list:       0-63
__output__ Vendor ID:                   ARM
__output__   Model name:                Neoverse-N1
__output__     Model:                   1
__output__     Thread(s) per core:      1
__output__     Core(s) per socket:      64
__output__     Socket(s):               1
__output__     Stepping:                r3p1
__output__     BogoMIPS:                243.75
__output__     Flags:                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimdd
__output__                             p
__output__ Caches (sum of all):
__output__   L1d:                       4 MiB (64 instances)
__output__   L1i:                       4 MiB (64 instances)
__output__   L2:                        64 MiB (64 instances)
__output__   L3:                        32 MiB (1 instance)
__output__ NUMA:
__output__   NUMA node(s):              1
__output__   NUMA node0 CPU(s):         0-63
```

On the Graviton4 instance, you see Neoverse-V2 cores instead:
```
__output__ Architecture:                aarch64
__output__   CPU op-mode(s):            64-bit
__output__   Byte Order:                Little Endian
__output__ CPU(s):                      64
__output__   On-line CPU(s) list:       0-63
__output__ Vendor ID:                   ARM
__output__   Model name:                Neoverse-V2
__output__     Model:                   1
__output__     Thread(s) per core:      1
__output__     Core(s) per socket:      64
__output__     Socket(s):               1
__output__     Stepping:                r0p1
__output__     BogoMIPS:                2000.00
__output__     Flags:                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc d
__output__                             cpop sha3 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svep
__output__                             mull svebitperm svesha3 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
__output__ Caches (sum of all):
__output__   L1d:                       4 MiB (64 instances)
__output__   L1i:                       4 MiB (64 instances)
__output__   L2:                        128 MiB (64 instances)
__output__   L3:                        36 MiB (1 instance)
__output__ NUMA:
__output__   NUMA node(s):              1
__output__   NUMA node0 CPU(s):         0-63
```

The two systems span two generations of Arm Neoverse server cores. Graviton2 uses Neoverse N1 cores with private 1 MB L2 caches per core and a 32 MB shared L3. Graviton4 uses Neoverse V2 cores with 2 MB private L2 caches per core and a 36 MB shared L3. These differences in cache sizes and memory technology directly impact the memory latency, bandwidth, and scaling behavior you observe in later benchmarks.

Notice that the Neoverse V2 `Flags` output is significantly longer than Neoverse N1. This reflects the newer Arm architecture: Neoverse V2 is based on Armv9, which adds extensions like SVE2, BF16, and I8MM that are absent on the Armv8-based Neoverse N1.

## Check DRAM configuration
Confirm the memory size with `free -h`:
```bash
free -h
```

Both systems have 128 GB RAM with similar output:
```
__output__ total        used        free      shared  buff/cache   available
__output__ Mem:           123Gi       1.6Gi       121Gi       1.2Mi       1.2Gi       121Gi
```

Graviton2 instances use DDR4 memory. Graviton4 instances use DDR5 memory, which provides higher bandwidth per channel than DDR4, but typically has slightly higher access latency in nanoseconds.

Both systems have a single NUMA node, meaning all cores have uniform access to all memory. On multi-socket Arm servers with multiple NUMA nodes, memory access latency depends on which node the data resides on. The ASCT `idle-latency` and `cross-numa-bandwidth` benchmarks are designed for multi-socket systems but will show minimal variation on single-node systems like these.

## Explore the core and cluster topology
Arm systems often organize cores into clusters that share a last-level cache. Understanding cluster boundaries is critical because latency and bandwidth behavior change when you cross them.

Use the extended `lscpu` output to see per-core details:
```bash
lscpu -e
```

The output is the same for both systems:
```
__output__ CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
__output__  0    0      0    0 0:0:0:0          yes
__output__  1    0      0    1 1:1:1:0          yes
__output__  2    0      0    2 2:2:2:0          yes
__output__  3    0      0    3 3:3:3:0          yes
...
__output__  63   0      0   63 63:63:63:0       yes
```

On both systems, every core has its own unique L1d, L1i, and L2 index, so those caches are private. The L3 index is 0 for all cores, confirming that the system exposes a single shared LLC across all cores, although it may be physically distributed across the interconnect on each system.

### Visualize the topology with hwloc
The `hwloc` package provides a visual representation of the hardware topology, including CPU cores, cache hierarchies, and NUMA nodes. The graphical output makes it easier to understand how cores are organized and which caches they share. Install it using the package manager:
```bash
sudo apt-get install -y hwloc
```

Generate a PNG diagram of the complete hardware hierarchy. The `hwloc-ls` command enumerates the system topology and formats it as a tree diagram showing cores, caches, and memory:
```bash
hwloc-ls --of png > topology.png
```

Here is the `hwloc` output from a Graviton2 c6g instance:

![hwloc topology for Graviton2 c6g](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/topology-c6g.webp)

Here is the `hwloc` output from a Graviton4 c8g instance:

![hwloc topology for Graviton4 c8g](https://learn.arm.com/learning-paths/servers-and-cloud-computing/memory-subsystem/topology-c8g.webp)

The diagrams show the full cache hierarchy at a glance: each core has its own private L1d, L1i, and L2 caches. On Graviton4, all cores share a single L3. On systems with more cores or multiple sockets, the tree branches further, making `hwloc` especially useful for spotting cluster and NUMA boundaries.

### Enumerate caches from sysfs
For detailed cache information, read the kernel’s `sysfs` entries directly.

Save the following script to a file named `cache.sh`:
```bash
for c in /sys/devices/system/cpu/cpu0/cache/index*; do
  echo "=== $(basename $c) ==="
  echo "Level: $(cat $c/level)"
  echo "Type:  $(cat $c/type)"
  echo "Size:  $(cat $c/size)"
  echo "Shared CPU list: $(cat $c/shared_cpu_list)"
  echo
done
```

Run the script on each instance:
```bash
bash ./cache.sh
```

On Graviton2 the output is:
```
__output__ === index0 ===
__output__ Level: 1
__output__ Type:  Data
__output__ Size:  64K
__output__ Shared CPU list: 0

__output__ === index1 ===
__output__ Level: 1
__output__ Type:  Instruction
__output__ Size:  64K
__output__ Shared CPU list: 0

__output__ === index2 ===
__output__ Level: 2
__output__ Type:  Unified
__output__ Size:  1024K
__output__ Shared CPU list: 0

__output__ === index3 ===
__output__ Level: 3
__output__ Type:  Unified
__output__ Size:  32768K
__output__ Shared CPU list: 0-63
```

On Graviton4 the output is:
```
__output__ === index0 ===
__output__ Level: 1
__output__ Type:  Data
__output__ Size:  64K
__output__ Shared CPU list: 0

__output__ === index1 ===
__output__ Level: 1
__output__ Type:  Instruction
__output__ Size:  64K
__output__ Shared CPU list: 0

__output__ === index2 ===
__output__ Level: 2
__output__ Type:  Unified
__output__ Size:  2048K
__output__ Shared CPU list: 0

__output__ === index3 ===
__output__ Level: 3
__output__ Type:  Unified
__output__ Size:  36864K
__output__ Shared CPU list: 0-63
```

Both systems have an `index3` entry showing a shared L3 cache. On Graviton2, the 32 MB L3 is shared across all 64 cores. On Graviton4, the 36 MB L3 is shared across all 64 cores. The `shared_cpu_list` of `0-63` confirms this.

## Record your system profile
Below is a summary of each system.

| Property             | Graviton2    | Graviton4    |
|----------------------|--------------|--------------|
| CPU model            | Neoverse N1  | Neoverse V2  |
| Core count           | 64           | 64           |
| L1D / L1I            | 64 KB / 64 KB| 64 KB / 64 KB|
| L2 (private)         | 1 MB         | 2 MB         |
| L3 (shared)          | 32 MB        | 36 MB        |
| DRAM type            | DDR4         | DDR5         |

## What you’ve learned and what’s next
In this section you:
- Identified the CPU model, core count, and NUMA topology of your test systems
- Explored the cache hierarchy and cluster layout using `lscpu`, `hwloc`, and `sysfs`
- Built a reference table of system properties for later comparison

The next section dives deeper into the cache hierarchy and explains what each level does and why the architectural choices matter for performance.