{"version":1,"type":"story","url":"https://digestai.news/story/openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users","json":"https://digestai.news/story/openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users.json","markdown":"https://digestai.news/story/openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users.md","slug":"openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users","headline":"OpenAI scales Azure Cosmos DB to serve 1 billion ChatGPT users","summary":"OpenAI has detailed how it scaled its internal storage platform, Habitat, to support over 1 billion users across 40 geographic regions. The system now manages more than 500 petabytes of data, with the largest single Azure Cosmos DB account handling over 20 million queries per second. The engineering team describes a complex journey from single-tenant to multi-tenant architectures, driven by the need to balance isolation with operational efficiency.\n\nInitially, OpenAI relied on a shared default container, which led to performance issues like noisy neighbors and throughput bottlenecks. The team then moved to separate containers for critical workloads, but this created significant capacity planning challenges for product teams. To resolve this, OpenAI collaborated closely with the Azure team to develop a new multi-tenant approach using co-managed containers. This included implementing request tiering and a new serverless mode that allows physical partitions to scale up to 10,000 request units per second.\n\nThe partnership also addressed reliability gaps exposed by a regional power outage in late 2024, which revealed that failover times for such massive accounts were untested and problematic. OpenAI and Azure now jointly monitor and test the infrastructure, sharing dashboards and engineering channels to manage failure modes at a scale where standard managed services are no longer sufficient.","keyPoints":["Habitat serves over 1 billion users and stores more than 500 petabytes of data.","The largest Azure Cosmos DB account handles over 20 million queries per second.","OpenAI and Azure co-developed a multi-tenant design with request tiering to prevent noisy neighbors."],"whyItMatters":"This demonstrates the engineering limits of standard cloud services at frontier AI scale. The collaboration model between OpenAI and Azure offers a blueprint for managing massive data loads and reliability in high-stakes AI infrastructure.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":["OpenAI","Azure"],"models":[],"people":[]},"firstPublishedAt":"2026-10-23T17:00:00Z","updatedAt":"2026-10-23T17:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"OpenAI","title":"Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)","url":"https://openai.com/index/scaling-storage-one-billion-users-part-two","publishedAt":"2026-10-23T17:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"OpenAI scales Azure Cosmos DB to serve 1 billion ChatGPT users\", 23 October 2026, https://digestai.news/story/openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users","publisher":"Digest AI","title":"OpenAI scales Azure Cosmos DB to serve 1 billion ChatGPT users","datePublished":"2026-10-23T17:00:00Z","url":"https://digestai.news/story/openai-scales-azure-cosmos-db-to-serve-1-billion-chatgpt-users"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}