# Amazon SageMaker adds training session APIs for custom LLM loops

Digest AI · Enterprise & Industry · published 2026-10-22T18:00:01Z · updated 2026-10-22T18:00:04Z

Canonical: https://digestai.news/story/amazon-sagemaker-adds-training-session-apis-for-custom-llm-loops

## Summary

Amazon SageMaker AI introduces training session APIs that let developers write their own training loops in Python while the service runs the GPU‑bound operations. The APIs expose core steps such as forwardbackward(), optimstep() and sample() as SageMaker calls, keeping the model loaded on managed accelerators and handling distribution, fault recovery and checkpointing.

Billing is per token processed, so only the GPU‑bound calls incur cost and idle‑GPU time is not charged. The blog shows a reinforcement‑learning example on the GSM8K math benchmark and a larger code‑generation task where a LoRA adapter for GPT‑OSS 20B is trained for 50 steps on 128 problems with eight responses each, raising reward from 0.46 to 0.60 and held‑out accuracy from 40% to 50.5%. A cookbook of ready‑to‑run recipes is released alongside the APIs.

The new feature targets teams that need algorithmic flexibility—custom loss functions, RL reward signals, multi‑stage pipelines—without provisioning GPU clusters. It fits between fully managed serverless model customization and self‑managed SageMaker training jobs, offering faster iteration and predictable spend for advanced LLM fine‑tuning and alignment research.

## Key points

- Training session APIs let users call forwardbackward, optimstep, and sample as SageMaker API operations, keeping the model loaded on GPUs.
- Billing is per token processed, so only GPU-bound calls incur cost, eliminating idle‑GPU charges.
- In a demo, a LoRA adapter for GPT‑OSS 20B trained 50 steps on 128 problems, raising reward from 0.46 to 0.60.

## Why it matters

The APIs give ML teams fine‑grained control over training algorithms while offloading GPU management, lowering costs and speeding experimentation for custom LLM fine‑tuning and RL workloads.

## Sources

1. [Write your own training loop — without managing GPUs — with Amazon SageMaker training session APIs](https://aws.amazon.com/blogs/machine-learning/write-your-own-training-loop-without-managing-gpus-with-amazon-sagemaker-training-sessions) (AWS Machine Learning Blog, 2026-10-22, primary source)
2. [Announcing new ways to customize models with SageMaker AI, without managing GPUs](https://aws.amazon.com/blogs/machine-learning/announcing-new-ways-to-customize-models-with-sagemaker-ai-without-managing-gpus) (AWS Machine Learning Blog, 2026-10-22, primary source)

Part of the developing story: [AWS Expands SageMaker AI Agent Capabilities](https://digestai.news/thread/aws-shows-how-contextual-bandits-lift-conversions-in-acquisition-funnels) (5 stories)

## Cite

Digest AI, "Amazon SageMaker adds training session APIs for custom LLM loops", 22 October 2026, https://digestai.news/story/amazon-sagemaker-adds-training-session-apis-for-custom-llm-loops

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/amazon-sagemaker-adds-training-session-apis-for-custom-llm-loops.json