{"version":1,"type":"story","url":"https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning","json":"https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning.json","markdown":"https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning.md","slug":"amazon-sagemaker-inference-supports-gpu-partitioning","headline":"Amazon SageMaker Inference supports GPU partitioning","summary":"Amazon SageMaker Inference now supports GPU partitioning via NVIDIA Multi‑Instance GPU (MIG), allowing users to split a single GPU into multiple isolated partitions. This lets more models run on a single accelerator, improving utilization and reducing cost.\n\nThe feature works on endpoints that use inference components. A partition receives a fixed share of compute units and memory, with hardware isolation. Customers can choose a fraction of a newer accelerator instead of a whole one. The article notes that provisioning with MIG adds roughly 17 seconds to endpoint startup, but otherwise the rest of the launch time is unchanged.\n\nSalesforce helped define requirements and validated the initial implementation. Richard Chang, Software Engineering Architect at Salesforce, said the capability would allow them to consolidate workloads safely, improve GPU utilization, reduce inference costs, and maintain predictable performance. The maximum number of partitions depends on the accelerator and never exceeds seven. MIG partitions a GPU into up to seven instances, each with its own compute, cache, memory and bandwidth.","keyPoints":["SageMaker now allows partitioning of GPUs using NVIDIA MIG, enabling multiple models per GPU","Partitioning can create up to seven partitions per GPU, reducing cost and improving isolation","Provisioning with MIG adds ~17 seconds to endpoint startup"],"whyItMatters":"GPU partitioning lets enterprises run more models per GPU, cutting inference costs and improving isolation, which benefits large‑scale AI deployments like those at Salesforce.","category":{"slug":"enterprise","name":"Enterprise & Industry","url":"https://digestai.news/category/enterprise"},"entities":{"companies":["Amazon Web Services","Salesforce"],"models":[],"people":["Richard Chang"]},"firstPublishedAt":"2026-11-06T18:01:00Z","updatedAt":"2026-11-06T18:01:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"AWS Machine Learning Blog","title":"SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints","url":"https://aws.amazon.com/blogs/machine-learning/sagemaker-inference-now-supports-multi-instance-gpu-to-maximize-gpu-utilization-for-model-endpoints","publishedAt":"2026-11-06T18:01:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Amazon SageMaker Inference supports GPU partitioning\", 6 November 2026, https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning","publisher":"Digest AI","title":"Amazon SageMaker Inference supports GPU partitioning","datePublished":"2026-11-06T18:01:00Z","url":"https://digestai.news/story/amazon-sagemaker-inference-supports-gpu-partitioning"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}