Amazon SageMaker adds training session APIs for custom LLM loops
Amazon SageMaker AI introduces training session APIs that let developers write their own training loops in Python while the service runs the GPU‑bound operations. The APIs expose core steps such as forwardbackward(), optimstep() and sample() as SageMaker calls, keeping the model loaded on managed accelerators and handling distribution, fault recovery and checkpointing.
Key points
- Training session APIs let users call forwardbackward, optimstep, and sample as SageMaker API operations, keeping the model loaded on GPUs.
- Billing is per token processed, so only GPU-bound calls incur cost, eliminating idle‑GPU charges.
- In a demo, a LoRA adapter for GPT‑OSS 20B trained 50 steps on 128 problems, raising reward from 0.46 to 0.60.
Billing is per token processed, so only the GPU‑bound calls incur cost and idle‑GPU time is not charged. The blog shows a reinforcement‑learning example on the GSM8K math benchmark and a larger code‑generation task where a LoRA adapter for GPT‑OSS 20B is trained for 50 steps on 128 problems with eight responses each, raising reward from 0.46 to 0.60 and held‑out accuracy from 40% to 50.5%. A cookbook of ready‑to‑run recipes is released alongside the APIs.
The new feature targets teams that need algorithmic flexibility—custom loss functions, RL reward signals, multi‑stage pipelines—without provisioning GPU clusters. It fits between fully managed serverless model customization and self‑managed SageMaker training jobs, offering faster iteration and predictable spend for advanced LLM fine‑tuning and alignment research.
The story so far
5 episodes →- Amazon SageMaker adds training session APIs for custom LLM loopsthis story
Write your own training loop — without managing GPUs — with Amazon SageMaker training session APIs
AWS Machine Learning Blog · 22 October 2026
Loading the full article…
This text was published by AWS Machine Learning Blog and written by Nicolas Jourdan. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
2sources- Announcing new ways to customize models with SageMaker AI, without managing GPUsPrimary source · AWS Machine Learning Blog ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Enterprise & Industry
All →- Morgan Stanley blocks third‑party AI agents after banks detect unauthorized usage · 2 src
- OpenAI and ICBA partner to give community banks AI cyber tools · 1 src
- IskayPet builds corporate knowledge assistant on Amazon Bedrock Managed Knowledge Bases · 1 src
- SpaceX launches Super Intelligence Partner Network with 130 firms including Nvidia · 1 src
- IBM launches Ready for SAP Solutions to modernize SMB operations · 1 src
Comments
via GitHub Discussions