Advance robotics reinforcement learning with Isaac Lab on DGX Spark
Introduction
Manipulate objects with a Franka 7-DOF robot arm
Train contact-rich manipulation policies with Isaac Lab on DGX Spark
Train multiple agents to coordinate two Shadow Hands in one simulation
Reproduce natural motion with Adversarial Motion Priors
Choose a reinforcement learning library for your task
Next Steps

  Log an issue

  Fork and edit

  Discuss on Discord

About this Learning Path

Skill level:  Advanced
Reading time:  2 hrs
Last updated:  18 Aug 2026
Skill level:
  Advanced
Reading time:
  2 hrs
Last updated:
  18 Aug 2026
Authors:Johnny Nunez, NVIDIA
Kieran Hejmadi, Arm
Odin Shen, Arm
Arm IP:Cortex-X Cortex-A
Tags: ML Linux Python Bash IsaacSim IsaacLab
Authors:
  • Johnny Nunez, NVIDIA
  • Kieran Hejmadi, Arm
  • Odin Shen, Arm
Arm IP:
Cortex-X Cortex-A
Tags:
ML Linux Python Bash IsaacSim IsaacLab

Who is this for?

This advanced topic is for robotics developers and AI researchers who want to extend an existing Isaac Sim and Isaac Lab setup on DGX Spark to manipulation, multi-agent training, and motion imitation.

What will you learn?

Upon completion of this Learning Path, you will be able to:

  • Train Franka manipulation policies with RSL-RL and RL Games
  • Compare Multi-Agent Proximal Policy Optimization (MAPPO) and Independent PPO (IPPO) in a two-agent Shadow Hand task
  • Train humanoid motion policies with Adversarial Motion Priors (AMP)
  • Select a reinforcement learning (RL) library that has a registered configuration for a task

Prerequisites

Before starting, you will need the following:

  • Access to an NVIDIA DGX Spark system with at least 50 GB of free disk space
  • Completion of the previous Isaac Sim and Isaac Lab Learning Path on the DGX Spark system
  • Experience with Python scripting
  • Basic understanding of reinforcement learning concepts such as rewards and policies

Summary

AI-assisted

This summary was drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
You’ll extend your Isaac Sim and Isaac Lab setup on DGX Spark to train increasingly complex reinforcement learning policies. First, you’ll train a Franka arm for manipulation and contact-rich tasks. Then, you’ll coordinate two Shadow Hands with MAPPO and IPPO, train humanoid motion with AMP, and select an RL library based on its registered task configuration and algorithm support.

Frequently asked questions

AI-assisted

These FAQs were drafted with an approved AI-assisted workflow and reviewed by Arm contributors before publication. Human technical review remains part of the process so the final page reflects engineering rigor, accuracy, and Arm editorial standards.

Close
?
How do I choose and stick to compatible Isaac Lab, Isaac Sim, and Python versions?
Use one compatible API set throughout training. Choose either Isaac Lab v2.3.2 with Isaac Sim 5.1.0 and Python 3.11, or Isaac Lab v3.0.0-beta2.patch1 with Isaac Sim 6.0.0 or 6.0.1 and Python 3.12.
How do I know the task and RL library I picked are supported?
Check the Isaac Lab environment list for supported task–library combinations. Start with a library that provides an upstream agent configuration for your task.
How do I verify that the trained policy opens the drawer?
Confirm that the arm aligns with the handle, moves the drawer along its rail after contact, and keeps the opening motion stable without slipping or shaking.
What changes when I switch between MAPPO and IPPO for the two Shadow Hands?
With MAPPO, the critic uses shared state during training while each hand acts from its own observations. With IPPO, each agent learns independently in the same environment.
What result should I expect from AMP training?
AMP uses reference motion‑capture data to guide learning toward smoother, more humanlike motion. Successful training produces trajectories that track the reference motions more closely than a standard RL policy.
Next