AI Models & Platforms
AWS Adds Managed Cluster Extensions to SageMaker HyperPod Slurm

Key PointsCluster extensions turn node software into declared state that HyperPod continuously…
View more
- Cluster extensions turn node software into declared state that HyperPod continuously validates and reconciles.
- Administrators can update settings or extension versions without replacing nodes or rerunning lifecycle scripts.
- Migrating existing clusters requires manually removing script-installed agents during a maintenance window.
Amazon Web Services launched managed cluster extensions for Amazon SageMaker HyperPod Slurm on November 20, 2026, bringing declarative configuration to software on running nodes through APIs or the SageMaker AI console. The first two extensions cover observability and user management, including directory integration.
The launch addresses the part of a shared AI training cluster that changes after provisioning. HyperPod had already added API-driven Slurm configuration, AMI-based node setup, continuous provisioning, automatic topology management and deep health checks during 2026. Those features manage the cluster’s definition, capacity, network layout and hardware health; cluster extensions apply the same approach to node software that changes as research teams, monitoring requirements and package versions evolve.
Desired State for Node Software
Administrators can install, configure, update and remove the AWS-managed packages through the console, AWS CLI or an SDK. HyperPod validates settings against the selected extension version’s schema before rollout and records the requested version and settings as the desired state. A node agent then applies only the components required by each node’s role, including on nodes added through scaling or replacement.
Updates submit only the settings being changed. HyperPod merges them into the stored configuration and restarts only affected components, without replacing nodes or rerunning lifecycle scripts. New extension versions can also be installed without an AMI update. HyperPod checks the stored settings against the new version’s schema before rollout, and upgrades only move forward.
Each extension reports Creating or Updating during rollout, Active when every node runs the requested configuration without a reported problem, and Degraded when nodes fail or report problems. DescribeClusterExtension groups node-level health issues by cause with a code, message and affected-node count. HyperPod retries internal failures; administrators can correct other causes and retry affected nodes with an update call containing no configuration changes.
Observability and User Management
The sagemaker-observability extension collects node, GPU, Elastic Fabric Adapter, Slurm job and Slurm cluster metrics, sending them to Amazon CloudWatch or Amazon Managed Service for Prometheus. Metric categories, levels and collection intervals are configurable, while nodes run only role-relevant components, such as DCGM and EFA exporters on GPU nodes and Slurm cluster metrics on the controller.
An optional Amazon Managed Grafana workspace adds pre-created Clusters and Jobs dashboards. They connect cluster-wide health and Slurm state with per-node infrastructure metrics and resource use for individual jobs, including CPU, memory, GPU utilization, disk I/O and NCCL data.
The sagemaker-user-management extension creates Linux users and groups with consistent IDs across nodes, controls SSH and passwordless sudo access by node type, and registers users with Slurm accounting. Accounts can be declared locally, drawn from Active Directory over LDAPS, or both, including connections to AWS Directory Service for Microsoft Active Directory. New and replacement nodes receive the same configuration, and administrators can review access through the API instead of signing into each node.
Migration, Permissions and Cost
Cluster extensions require a HyperPod Slurm cluster using the November 2, 2026 HyperPod AMI or later, with continuous provisioning enabled. Because extensions run software as root, AWS treats create, update and delete permissions as administrator permissions and advises against granting them to the cluster execution role. That role still needs access to metric destinations and Active Directory secrets, while user management requires a shared file system for home directories. Separate extension ARNs allow platform and monitoring teams to own different extensions.
Existing clusters can migrate one capability at a time while scripts and extensions coexist. The process requires updating the cluster software, disabling that capability in lifecycle scripts, manually removing script-installed agents during a maintenance window, and creating the extension on the clean cluster. Each component needs one owner: managing the same exporter, user or service through both mechanisms can cause installation or runtime conflicts. Custom AMIs and lifecycle scripts remain available for organization-specific software, and administrators retain SSH and direct configuration access to every node, including the controller.
Both extensions are available in all Regions where SageMaker HyperPod is supported. There is no additional charge for cluster extensions, but customers pay for underlying resources such as CloudWatch, Amazon Managed Service for Prometheus and AWS Secrets Manager.
For shared foundation-model clusters, the operational consequence is that common node software becomes a declared, continuously checked part of cluster state rather than a collection of provisioning scripts. Removing script-installed agents is the one manual fleet-wide migration step; afterward, scale-ups and replacement nodes inherit the same extension settings, while organization-specific tooling remains under the cluster team’s control.












