# Amazon SageMaker Inference supports GPU partitioning

Digest AI · Enterprise & Industry · published 2026-11-06T18:01:00Z

Canonical: https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning

## Summary

Amazon SageMaker Inference now supports GPU partitioning via NVIDIA Multi‑Instance GPU (MIG), allowing users to split a single GPU into multiple isolated partitions. This lets more models run on a single accelerator, improving utilization and reducing cost.

The feature works on endpoints that use inference components. A partition receives a fixed share of compute units and memory, with hardware isolation. Customers can choose a fraction of a newer accelerator instead of a whole one. The article notes that provisioning with MIG adds roughly 17 seconds to endpoint startup, but otherwise the rest of the launch time is unchanged.

Salesforce helped define requirements and validated the initial implementation. Richard Chang, Software Engineering Architect at Salesforce, said the capability would allow them to consolidate workloads safely, improve GPU utilization, reduce inference costs, and maintain predictable performance. The maximum number of partitions depends on the accelerator and never exceeds seven. MIG partitions a GPU into up to seven instances, each with its own compute, cache, memory and bandwidth.

## Key points

- SageMaker now allows partitioning of GPUs using NVIDIA MIG, enabling multiple models per GPU
- Partitioning can create up to seven partitions per GPU, reducing cost and improving isolation
- Provisioning with MIG adds ~17 seconds to endpoint startup

## Why it matters

GPU partitioning lets enterprises run more models per GPU, cutting inference costs and improving isolation, which benefits large‑scale AI deployments like those at Salesforce.

## Sources

1. [SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints](https://aws.amazon.com/blogs/machine-learning/sagemaker-inference-now-supports-multi-instance-gpu-to-maximize-gpu-utilization-for-model-endpoints) (AWS Machine Learning Blog, 2026-11-06, primary source)

## Cite

Digest AI, "Amazon SageMaker Inference supports GPU partitioning", 6 November 2026, https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning.json