Alibaba Releases Open‑Weight Qwen3.8‑2.4T‑A95B; Deploys on SageMaker HyperPod
Alibaba’s Qwen team unveiled the Qwen3.8‑2.4T‑A95B model on August 12, 2026, marking the first open‑weight release of a Qwen‑Max‑class model. The 2.4 trillion‑parameter network activates 95 billion parameters per token, supports a native context window of 262 k tokens (expandable to 1 M), and is engineered for demanding agentic and reasoning tasks such as multi‑step coding, long‑horizon…
Key points
- Qwen3.8‑2.4T‑A95B is a 2.4 trillion‑parameter open‑weight LLM with 262 k‑token context
- Deployment uses SageMaker HyperPod’s ml.p6‑b300 (8× B300 GPUs) with vLLM and NVFP4 quantization
- Model supports built‑in reasoning, tool calling, and Multi‑Token Prediction for efficient agentic workloads
The post details how to run the model on Amazon SageMaker HyperPod using vLLM on an ml.p6‑b300 instance (8× NVIDIA B300 Blackwell Ultra GPUs). It covers cluster provisioning, vLLM configuration (NVFP4 quantization, Multi‑Token Prediction, built‑in reasoning controls), and the HyperPod inference operator that manages model download, scheduling, autoscaling, and health monitoring.
With open weights, users gain full control over data residency and inference behavior, eliminating per‑token API fees. The deployment guide demonstrates that a single 8‑GPU node can host the compressed 1.2 TB model, making frontier‑scale inference feasible for enterprises that prefer self‑hosted solutions.
The story so far
10 episodes →- Alibaba Releases Open‑Weight Qwen3.8‑2.4T‑A95B; Deploys on SageMaker HyperPodthis story
Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM
AWS Machine Learning Blog · 9 September 2026
Loading the full article…
This text was published by AWS Machine Learning Blog and written by Dmitry Soldatkin. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- Liquid AI releases d1-ultra decision model at $0.042 per million tokens · 1 src
- Meta releases Muse Spark Open, an 816B-parameter open-weight multimodal model with 42B active parameters and 1M-token context · 21 src
- Google launches Gemini 4 Argon model globally for Pro and Ultra subscribers with introductory API pricing · 32 src
- Microsoft AI chief says in-house model beats OpenAI's using quarter of compute · 1 src
- NVIDIA releases Nemotron 3.5 Super VL and Bolt edge models for enterprise agents · 8 src
Comments
via GitHub Discussions