Who is this for?
This is an introductory topic for developers, data engineers, and platform engineers who want to build high-performance data pipelines and analytics systems using Alluxio on Arm-based cloud environments.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Install and configure Alluxio on Azure Cobalt 100 Arm64 virtual machines
- Configure data caching using Alluxio memory storage
- Integrate Alluxio with Apache Spark for analytics workloads
- Benchmark data access performance and understand caching benefits
Prerequisites
Before starting, you will need the following:
- A
Microsoft Azure account
with access to Cobalt 100 based instances (Dpsv6)
- Basic knowledge of Linux command-line operations
- Familiarity with SSH and remote server access
- Basic understanding of data processing, storage systems, and caching concepts
Summary
AI-assisted
This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication.
Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
You’ll deploy Alluxio on an Arm-based virtual machine powered by Azure Cobalt 100 and use it as a data orchestration and caching layer for Apache Spark. After creating a Dpsv6 VM, you’ll enable the Alluxio Web UI by adding an inbound rule for TCP port 19999 in the Azure Network Security Group. You’ll install and configure Alluxio with local storage and memory-backed caching, then install Apache Spark under /opt and configure Spark to read through Alluxio. The result is a set of read workloads comparing uncached and cached access that you can use to interpret the impact of in-memory caching on analytics tasks.
Frequently asked questions
AI-assisted
These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication.
Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.
Use a general-purpose VM in the Dpsv6 series to run on Azure Cobalt 100 Arm-based processors. The creation steps focus on this series.
How do I allow access to the Alluxio Web UI from the internet?Add an inbound rule for TCP port 19999 in the Azure Network Security Group (NSG). You can attach the NSG to the VM’s network interface or its subnet.
How can I confirm the Alluxio Web UI is reachable after opening the port?After creating the NSG rule for port 19999, browse to the VM’s public IP using that port to load the Alluxio Web UI. If it doesn’t load, verify the NSG is correctly attached and the rule allows inbound traffic.
What preparation should I do on the VM before installing Alluxio?Update the package index, apply available updates, and install basic tools such as wget, curl, and tar. These tools download and extract the required software.
Where should Apache Spark be installed and how is it used with Alluxio?Install Apache Spark and place it under /opt as shown in the Learning Path. Spark is then configured to read through Alluxio, so frequently accessed data can be cached in memory.