Deploy ML models to Arm edge devices using Edge Impulse and AWS IoT Greengrass
Introduction
Understand the Edge Impulse and AWS IoT Greengrass deployment architecture
Select and set up your edge device
Set up your Edge Impulse project
Install AWS IoT Greengrass on your Arm edge device
Store your Edge Impulse API key in AWS Secrets Manager
Create the Edge Impulse Greengrass component
Deploy the component to your edge device
Verify inference and view results
Next Steps
Deploy ML models to Arm edge devices using Edge Impulse and AWS IoT Greengrass
Introduction
Understand the Edge Impulse and AWS IoT Greengrass deployment architecture
Select and set up your edge device
Set up your Edge Impulse project
Install AWS IoT Greengrass on your Arm edge device
Store your Edge Impulse API key in AWS Secrets Manager
Create the Edge Impulse Greengrass component
Deploy the component to your edge device
Verify inference and view results
Next Steps
Prepare an NVIDIA Jetson board
NVIDIA Jetson boards such as Nano, Xavier, and Orin provide GPU-accelerated inference for Edge Impulse models. You’ll complete prerequisites, dependency installation, and the component configuration for running the Edge Impulse Linux Runner on a Jetson device with AWS IoT Greengrass.
Prerequisites
Before you begin, make sure you have:
- An NVIDIA Jetson board with a power supply
- JetPack 5.x or 6.0 already flashed onto the device
- A Network connection (Ethernet or Wi-Fi) and SSH access to the device
- (Optional) A USB camera for live inference; without a camera, the Edge Impulse Linux Runner uses a sample video file
Verify JetPack version
After booting the Jetson, confirm which JetPack version is installed:
cat /etc/nv_tegra_release
The output should indicate L4T (Linux for Tegra) version 34.x or later for JetPack 5.x, or version 36.x for JetPack 6.0.
Connect over SSH
If you haven’t already, connect to the Jetson from your local machine. Replace the placeholder with the device’s IP address:
ssh your-username@<your-jetson-ip-address>
If you’re not sure of the IP address, check your router’s admin page for connected devices. If you have a monitor connected, run hostname -I on the Jetson.
Install dependencies
Update the package list and install the build tools, Node.js, and GStreamer plugins that the Edge Impulse Linux Runner requires:
sudo apt update
sudo apt install -y curl unzip
sudo apt install -y gcc g++ make build-essential nodejs sox gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-base gstreamer1.0-plugins-base-apps
Greengrass Nucleus Classic is Java-based, so you also need to install a JDK:
sudo apt install -y default-jdk
Install any available security updates:
sudo apt upgrade -y
(Optional) Verify the camera
If you have a USB camera connected, confirm that the system detects it:
ls /dev/video*
The output is similar to:
/dev/video0
If nothing appears, check that the camera is plugged in securely and try a different USB port.
Save the component configuration
The following JSON configurations set up the Edge Impulse Greengrass component for the Jetson. Choose the configuration that matches your setup and save it to a text file on your local machine. You’ll paste it into the Greengrass deployment configuration in a later step.
If your device is running JetPack 6.x or later, the render group is required for the Greengrass service user to access the GPU. Both JSON configurations already include render in the ei_ggc_user_groups field. If you’re running JetPack 5.x, you can remove render from that field, though leaving it in place doesn’t cause issues.
With a USB camera
This configuration captures live video from /dev/video0 at 640 × 480 resolution. The --force-variant float32 flag selects the float32 model variant. --silent suppresses console output because the Edge Impulse Linux Runner runs as a background service:
{
"Parameters": {
"node_version": "20.18.2",
"vips_version": "8.12.1",
"device_name": "MyNvidiaJetsonEdgeDevice",
"launch": "runner",
"sleep_time_sec": 10,
"lock_filename": "/tmp/ei_lockfile_runner",
"gst_args": "v4l2src:device=/dev/video0:!:video/x-raw,width=640,height=480:!:videoconvert:!:jpegenc",
"eiparams": "--greengrass",
"iotcore_backoff": "-1",
"iotcore_qos": "1",
"ei_bindir": "/usr/local/bin",
"ei_sm_secret_id": "EI_API_KEY",
"ei_sm_secret_name": "ei_api_key",
"ei_poll_sleeptime_ms": 2500,
"ei_local_model_file": "/home/ggc_user/data/currentModel.eim",
"ei_shutdown_behavior": "__none__",
"ei_ggc_user_groups": "video audio input users system render",
"install_kvssink": "no",
"publish_inference_base64_image": "no",
"enable_cache_to_file": "no",
"cache_file_directory": "__none__",
"enable_threshold_limit": "no",
"metrics_sleeptime_ms": 30000,
"default_threshold": 65.0,
"threshold_criteria": "ge",
"enable_cache_to_s3": "no",
"s3_bucket": "__none__"
}
}
Without a camera
This configuration reads inference input from a local sample video file. The ei_local_model_file field points to a pre-downloaded model. ei_shutdown_behavior is set to wait_on_restart so that the Edge Impulse Linux Runner pauses after the video ends and waits for a restart command:
{
"Parameters": {
"node_version": "20.18.2",
"vips_version": "8.12.1",
"device_name": "MyNvidiaJetsonEdgeDevice",
"launch": "runner",
"sleep_time_sec": 10,
"lock_filename": "/tmp/ei_lockfile_runner",
"gst_args": "filesrc:location=/home/ggc_user/data/testSample.mp4:!:decodebin:!:videoconvert:!:videorate:!:video/x-raw,framerate=2200/1:!:jpegenc",
"eiparams": "--greengrass",
"iotcore_backoff": "-1",
"iotcore_qos": "1",
"ei_bindir": "/usr/local/bin",
"ei_sm_secret_id": "EI_API_KEY",
"ei_sm_secret_name": "ei_api_key",
"ei_poll_sleeptime_ms": 2500,
"ei_local_model_file": "/home/ggc_user/data/currentModel.eim",
"ei_shutdown_behavior": "wait_on_restart",
"ei_ggc_user_groups": "video audio input users system render",
"install_kvssink": "no",
"publish_inference_base64_image": "no",
"enable_cache_to_file": "no",
"cache_file_directory": "__none__",
"enable_threshold_limit": "no",
"metrics_sleeptime_ms": 30000,
"default_threshold": 50,
"threshold_criteria": "ge",
"enable_cache_to_s3": "no",
"s3_bucket": "__none__"
}
}
When running a model compiled specifically for a Jetson GPU, the first invocation can take 2-3 minutes while the model loads into GPU memory. Subsequent invocations are much faster.
What you’ve accomplished and what’s next
You’ve prepared your NVIDIA Jetson, installed its dependencies, and saved the component configuration for your selected input source.
Your NVIDIA Jetson is ready. Next, you’ll set up the Edge Impulse project .