{"version":1,"type":"story","url":"https://digestai.news/story/deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat","json":"https://digestai.news/story/deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat.json","markdown":"https://digestai.news/story/deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat.md","slug":"deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat","headline":"DeepSeek’s inference costs 12x more than training, YC Paper Club estimates","summary":"A Y Combinator Paper Club session compared training and inference costs using DeepSeek’s public data. The group found that while training DeepSeek V3 took about **2.8 million H800 hours**, daily inference already consumes **43,000 H800 hours**—and could be higher. Over two years, inference’s annual compute volume becomes **12 times** that of training, the session noted. The analysis highlights that inference efficiency drives profitability: DeepSeek’s profit margin is estimated at **80%**, partly due to innovations in inference optimization like **Mixture of Experts (MoE)** and **RadixAttention**—a technique that reuses shared prefixes to cut costs by up to **56%** in latency, according to SGLang’s Liansheng Yin.","keyPoints":["DeepSeek’s training used 2.8M H800 hours; inference now runs 6x that daily, 12x annually","Inference efficiency boosts margins: DeepSeek claims **80% profit** from inference optimizations like MoE and RadixAttention","Open models (e.g., Together AI’s **400T tokens/month**) cut costs by **10x** but vary in quality due to quantization or engine differences"],"whyItMatters":"Inference costs dominate AI’s operational budget, yet most research focuses on training. This shift demands better tools—like Together AI’s compiler or SGLang’s caching—to cut latency and costs, which could redefine cloud AI economics.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["DeepSeek","Y Combinator","Together AI","Ollama","Luminal","SGLang"],"models":["DeepSeek V3","Kimi K3","Kimi K2","M3","GPT-6","GPT-7"],"people":["Dan Fu","Jeff Morgan","Joe Fiotti","Liansheng Yin"]},"firstPublishedAt":"2026-10-26T04:00:00Z","updatedAt":"2026-10-26T04:00:00Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"note.com","title":"\"Inference is 12 times more important than training\" — YC Paper Club estimates based on DeepSeek's public figures: training takes about 2.8 million H800 hours, while","url":"https://note.com/startup_now0708/n/ne910ada7f8f9?hl=en","publishedAt":"2026-10-26T04:00:00Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[],"thread":{"title":"DeepSeek AI Model Saga From Open Source To Benchmark Wins","url":"https://digestai.news/thread/deepseek-open-sources-deepinfer-stack-potentially-lowering-inference-costs","storyCount":4},"cite":{"text":"Digest AI, \"DeepSeek’s inference costs 12x more than training, YC Paper Club estimates\", 26 October 2026, https://digestai.news/story/deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat","publisher":"Digest AI","title":"DeepSeek’s inference costs 12x more than training, YC Paper Club estimates","datePublished":"2026-10-26T04:00:00Z","url":"https://digestai.news/story/deepseeks-inference-costs-12x-more-than-training-yc-paper-club-estimat"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}