Deploy Ceph RADOS Block Device storage on an Arm AGI CPU host
Introduction
Understand the staged Ceph topology
Create three virtual machines
Prepare and validate the virtual machines
Bootstrap Ceph on one node
Create and use an RBD volume
Expand Ceph to three nodes
Test Ceph RBD during an OSD outage and recovery
Next Steps
Deploy Ceph RADOS Block Device storage on an Arm AGI CPU host
Create the storage cluster
You’ll bootstrap a new Ceph cluster on ceph1 and provision its data disk as the first object storage daemon (OSD).
Run all commands inside ceph1 to create a working single-node cluster, ready to store an RADOS Block Device (RBD) image.
You’ll add the other two VMs later.
Install cephadm
Use the
standalone cephadm installation method
with release 19.2.6. You’ll use Ceph Squid 19.2.6, but the Ubuntu 26.04 repositories provide Ceph 20.2.0. To keep the deployment tools and cluster on the same release, download cephadm 19.2.6 directly rather than installing it through the Ubuntu package manager (APT). Use the pinned Ceph 19.2.6 container image.
Download the executable and check its version before installing it:
(
set -e
mkdir -p "$HOME/ceph-tools"
cd "$HOME/ceph-tools"
curl --fail --show-error --location \
--output cephadm-19.2.6 \
https://download.ceph.com/rpm-19.2.6/el9/noarch/cephadm
python3 ./cephadm-19.2.6 version
)
Install the executable and check the host:
sudo install -m 755 "$HOME/ceph-tools/cephadm-19.2.6" /usr/local/sbin/cephadm
sudo /usr/local/sbin/cephadm version
sudo /usr/local/sbin/cephadm check-host
The expected output is:
cephadm version 19.2.6 (f9fd95b4335bad6a26d7a74468f55d269e8dbef4) squid (stable)
podman (/usr/bin/podman) version 5.7.0 is present
systemctl is present
lvcreate is present
Unit chrony.service is enabled and running
Host looks OK
Bootstrap the cluster
Set CEPH1_IP to the stable address that you recorded earlier for ceph1:
CEPH1_IP=192.168.100.11
CEPH_IMAGE='quay.io/ceph/ceph@sha256:ff836bb28e7d0be0ec4dd22e79407174452a96e11ee62e9cdd1fa47f4bcc3bdc'
sudo podman pull "$CEPH_IMAGE"
sudo podman run --rm --entrypoint /usr/bin/ceph "$CEPH_IMAGE" --version
sudo /usr/local/sbin/cephadm --image "$CEPH_IMAGE" bootstrap --mon-ip "$CEPH1_IP"
Bootstrap can take several minutes while Ceph downloads container images and starts the first monitor and manager.
You should see the message Bootstrap complete.
At completion, cephadm writes the cluster configuration and administrator keyring under /etc/ceph/.
Run the administrative commands through cephadm shell to use the Ceph client from the cluster’s container image:
sudo /usr/local/sbin/cephadm shell -- ceph --version
sudo /usr/local/sbin/cephadm shell -- ceph -s
sudo /usr/local/sbin/cephadm shell -- ceph versions
sudo /usr/local/sbin/cephadm shell -- ceph orch ps
The shell prints the inferred cluster ID, configuration path, and container image before the command output. Compare the container client version with the monitor and manager versions reported by ceph versions.
To complete the Learning Path, use the pinned container client for administration.
Squid 19.2.6 introduced the aes256k CephX key type. An older userspace client can fail to parse these credentials even when the guest kernel supports them. Upstream kernel support began with Linux 7.0. For more information, see the
Ceph key-format compatibility guidance
.
The status initially reports no OSDs. A warning about the default pool replication count is expected until you add storage.
Add the first OSD
Ask the orchestrator whether /dev/vdb is available:
sudo /usr/local/sbin/cephadm shell -- ceph orch device ls --hostname ceph1 --wide
The output is similar to:
HOST PATH TYPE TRANSPORT RPM DEVICE ID SIZE HEALTH IDENT FAULT AVAILABLE REFRESHED REJECT REASONS
ceph1 /dev/sr0 hdd QEMU_CD-ROM_drive-scsi0-0-0-0 368k N/A N/A No 11m ago Has a FileSystem, Insufficient space (<5GB), read-only
ceph1 /dev/vdb hdd 10.0G N/A N/A Yes 11m ago
The seed ISO size and refresh times can vary.
For the VMs that you created, the device listing should include the following size values:
| Device | Purpose | Expected availability |
|---|---|---|
/dev/sr0 | Cloud-init seed CD-ROM | No: it contains a file system, is read-only, and is smaller than 5 GB. |
/dev/vdb | Empty 10 GB Ceph data disk | Yes, with no reject reasons. |
No availability for /dev/sr0 is expected. Keep the seed ISO attached and leave it out of OSD provisioning.
Confirm that the unused data disk is available. For this Learning Path, the example unused disk is /dev/vdb. If you used a different disk earlier, replace /dev/vdb in the command with your chosen device name.
Give the complete disk to Ceph:
sudo /usr/local/sbin/cephadm shell -- ceph orch daemon add osd ceph1:/dev/vdb
This operation erases and provisions the unused data disk. Wait for the OSD to start, then inspect the result:
sudo /usr/local/sbin/cephadm shell -- ceph orch ps --hostname ceph1 --daemon-type osd
sudo /usr/local/sbin/cephadm shell -- ceph osd tree
The ceph orch ps command reports a running OSD, similar to:
NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID
osd.0 ceph1 running (37s) 35s ago 37s 10.4M 4096M 19.2.6 9e014221cd72 a97627c1ce67
The following values can vary:
- Times
- Memory usage
- Version
- Image ID
- Container ID
The output of the ceph osd tree command is similar to:
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 0.00980 root default
-3 0.00980 host ceph1
0 hdd 0.00980 osd.0 up 1.00000 1.00000
The tree IDs and weight can vary with the cluster and disk capacity.
The output includes one OSD beneath host ceph1, with status up.
What you’ve accomplished and what’s next
You’ve bootstrapped a Ceph monitor and manager and added one OSD on ceph1.
Next, you’ll create an RBD image and use it as a Linux block device. You’ll test the complete client path: the pinned userspace tools, the Ubuntu 26.04 guest kernel, authentication, and persistent filesystem access.