{"version":1,"type":"story","url":"https://digestai.news/story/ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack","json":"https://digestai.news/story/ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack.json","markdown":"https://digestai.news/story/ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack.md","slug":"ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack","headline":"IBM and Red Hat launch llm-d 1.0 distributed inference stack","summary":"IBM Research announced llm-d 1.0 at the PyTorch Conference North America on October 20, 2026. The Kubernetes‑native stack is positioned as a vendor‑neutral way for organizations to run any model on any accelerator in any cloud, aiming to curb rising inference costs. Lead maintainers include IBM, Red Hat and Google, with CoreWeave and NVIDIA co‑founding the project and contributors such as AMD, Cisco, Hugging Face, Intel, Lambda and Mistral AI. Carlos Costa, a Distinguished Engineer at IBM Research, highlighted multi‑tenant quality‑of‑service features, zero‑downtime upgrades and a stable API.\n\nUsing two 64‑GPU H200 layouts, the stack cost $0.05‑$0.08 per million input tokens and $1.92‑$3.55 per million output tokens, covering GPU rental only. A separate demo on 544 NVIDIA H100 GPUs delivered over 6.6 million output tokens per minute at peak, serving up to 3,000 concurrent coding agents without preemption. IBM and Red Hat claim self‑hosting with llm-d can be five‑to‑ten times cheaper per token than commercial API pricing.","keyPoints":["IBM, Red Hat and Google lead the open‑source llm‑d 1.0 inference stack released Oct 20, 2026","Benchmarks report $0.05‑$0.08 per M input tokens and $1.92‑$3.55 per M output tokens on H200 GPUs","Deployment on 544 H100 GPUs served 3,000 agents, delivering >6.6 M tokens/min with zero preemptions"],"whyItMatters":"A production‑grade, cloud‑agnostic inference stack lets enterprises host large models in‑house, cutting costs and reducing reliance on proprietary API providers.","category":{"slug":"enterprise","name":"Enterprise & Industry","url":"https://digestai.news/category/enterprise"},"entities":{"companies":["IBM","Red Hat","Google","NVIDIA","AMD","Intel"],"models":["GLM-5.2","vLLM","SGLang"],"people":["Carlos Costa","Stephen Watt","Maroon Ayoub"]},"firstPublishedAt":"2026-10-20T12:37:09Z","updatedAt":"2026-10-20T12:37:09Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"Unite.AI","title":"Llm-D 1.0 Debuts at PyTorch Conference with Production Inference Stack","url":"https://unite.ai/llm-d-1-0-debuts-at-pytorch-conference-with-production-inference-stack","publishedAt":"2026-10-20T12:37:09Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"IBM and Red Hat launch llm-d 1.0 distributed inference stack\", 20 October 2026, https://digestai.news/story/ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack","publisher":"Digest AI","title":"IBM and Red Hat launch llm-d 1.0 distributed inference stack","datePublished":"2026-10-20T12:37:09Z","url":"https://digestai.news/story/ibm-and-red-hat-launch-llm-d-1-0-distributed-inference-stack"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}