Run Spark applications on Microsoft Azure Cobalt 100 processors
Introduction
Getting started with Microsoft Azure Cobalt 100, Azure Linux 3.0, and Apache Spark
Create an Azure Cobalt 100 Arm64 virtual machine
Set up an Azure Linux 3.0 environment
Install Apache Spark on Azure Cobalt 100 processors
Validate Apache Spark on Azure Cobalt 100 Arm64 VMs
Benchmark Apache Spark
Next Steps
Run Spark applications on Microsoft Azure Cobalt 100 processors
Introduction
Getting started with Microsoft Azure Cobalt 100, Azure Linux 3.0, and Apache Spark
Create an Azure Cobalt 100 Arm64 virtual machine
Set up an Azure Linux 3.0 environment
Install Apache Spark on Azure Cobalt 100 processors
Validate Apache Spark on Azure Cobalt 100 Arm64 VMs
Benchmark Apache Spark
Next Steps
Who is this for?
This is an advanced topic that introduces Spark deployment on Microsoft Azure Cobalt 100 (Arm-based) virtual machines. It is designed for developers migrating Spark applications from x86_64 to Arm.
What will you learn?
Upon completion of this Learning Path, you will be able to:
- Provision an Azure Arm64 virtual machine using Azure console
- Learn how to create an Azure Linux 3.0 Docker container
- Deploy a Spark application inside an Azure Linux 3.0 Arm64-based Docker container or an Azure Linux 3.0 custom-image based Azure virtual machine
- Run a suite of Spark benchmarks to understand and evaluate performance on the Azure Cobalt 100 virtual machine
Prerequisites
Before starting, you will need the following:
- A Microsoft Azure account with access to Cobalt 100 based instances (Dpsv6)
- A machine with Docker installed
- Familiarity with distributed computing concepts and the Apache Spark architecture