{"version":1,"type":"story","url":"https://digestai.news/story/aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks","json":"https://digestai.news/story/aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks.json","markdown":"https://digestai.news/story/aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks.md","slug":"aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks","headline":"AWS boosts MoE RL training throughput by 40% with EFA and DeepEP on EKS","summary":"AWS outlines an architecture combining **Amazon EKS**, **Elastic Fabric Adapter (EFA)**, and **DeepEP** to accelerate **Mixture-of-Experts (MoE)** reinforcement learning (RL) training. The setup addresses three key challenges: balancing rollout generation and policy training, managing heterogeneous compute and memory demands, and optimizing communication between nodes. MoE models, which use sparsity to scale large language models, introduce unique training bottlenecks due to dynamic **Expert Parallelism (EP)**—a form of all-to-all token routing across accelerators that becomes more inter-node as models grow larger.\n\nAWS claims the new approach delivers **40% higher aggregate RL rollout throughput** on **48 P5en instances** (16 for training, 32 for inference) compared to a baseline Slime stack. The improvement comes from replacing generic **NCCL all-to-all collectives** with **DeepEP’s topology-aware kernels**, which route tokens via **NVLink for intra-node** and **EFA for inter-node** communication. The system also leverages **Amazon EC2 Spot Instances** for cost-efficient rollout generation, isolating policy training workers from interruptions. The blog post provides step-by-step setup instructions, including EKS cluster configuration, EFA plugin deployment, and job submission via **TorchX**. AWS emphasizes modularity: components like rollout generation, policy training, and storage can scale independently, reducing idle capacity and cost.","keyPoints":["AWS’s architecture combines EKS, EFA, and DeepEP to cut MoE RL training bottlenecks by 40% on 48 P5en instances","DeepEP replaces generic NCCL collectives with NVLink/EFA-optimized kernels for sparse, dynamic MoE token routing","Spot Instances reduce rollout generation costs by tolerating interruptions without stalling tightly coupled policy training"],"whyItMatters":"This optimization directly impacts cloud providers and AI labs running large-scale MoE RL training, lowering costs and speeding iteration. For enterprises, it validates AWS’s infrastructure as a scalable solution for next-gen sparse models like those from Mistral or DeepSeek, where communication overhead is a growing constraint.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["AWS","Amazon"],"models":[],"people":[]},"firstPublishedAt":"2026-09-25T16:29:50Z","updatedAt":"2026-09-25T16:29:50Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"AWS Machine Learning Blog","title":"Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput","url":"https://aws.amazon.com/blogs/machine-learning/scaling-moe-reinforcement-learning-on-amazon-eks-with-efa-and-deepep-with-40-more-throughput","publishedAt":"2026-09-25T16:29:50Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"AWS Expands SageMaker AI Capabilities","url":"https://digestai.news/thread/aws-adds-whisperx-to-sagemaker-for-speaker-labeled-audio-transcription","storyCount":3},"cite":{"text":"Digest AI, \"AWS boosts MoE RL training throughput by 40% with EFA and DeepEP on EKS\", 25 September 2026, https://digestai.news/story/aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks","publisher":"Digest AI","title":"AWS boosts MoE RL training throughput by 40% with EFA and DeepEP on EKS","datePublished":"2026-09-25T16:29:50Z","url":"https://digestai.news/story/aws-boosts-moe-rl-training-throughput-by-40-with-efa-and-deepep-on-eks"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}