Run Spark applications on Microsoft Azure Cobalt 100 processors

About this Learning Path

Who is this for?

This is an advanced topic that introduces Spark deployment on Microsoft Azure Cobalt 100 (Arm-based) virtual machines. It is designed for developers migrating Spark applications from x86_64 to Arm.

What will you learn?

Upon completion of this Learning Path, you will be able to:

• Provision an Azure Arm64 virtual machine using Azure console
• Learn how to create an Azure Linux 3.0 Docker container
• Deploy a Spark application inside an Azure Linux 3.0 Arm64-based Docker container or an Azure Linux 3.0 custom-image based Azure virtual machine
• Run a suite of Spark benchmarks to understand and evaluate performance on the Azure Cobalt 100 virtual machine

Prerequisites

Before starting, you will need the following:

• A Microsoft Azure account with access to Cobalt 100 based instances (Dpsv6)
• A machine with Docker installed
• Familiarity with distributed computing concepts and the Apache Spark architecture