{"version":1,"type":"story","url":"https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2","json":"https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2.json","markdown":"https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2.md","slug":"deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2","headline":"Deepgram improves Nova-3 speech-to-text efficiency on AWS trainium2","summary":"Deepgram processes quadrillions of audio tokens a year for more than 200,000 developers and 1,400 organizations. To cut inference costs, the company ported its Nova-3 speech‑to‑text model to AWS Trainium2 using TorchNeuron, replacing several layers with pre‑built kernels from the public NKI Library and tuning them with Neuron Explorer and the Neuron Agentic Development toolkit. The final setup runs eight model replicas on a single EC2 trn2.3xlarge instance.\n\nIn offline, throughput‑oriented batch workloads the Trainium2 configuration delivered 11–70 percent more audio per dollar than Deepgram’s prior compute options, based on instance pricing at the time of evaluation. A custom Rust data‑parallel runtime eliminated Python interpreter overhead and added a 7.5 percent throughput gain over a multiprocessing Python baseline. These improvements lower costs for Deepgram’s customers and demonstrate that AWS Trainium can be an efficient platform for large‑scale voice AI inference.","keyPoints":["Deepgram’s Nova-3 runs eight replicas on a single trn2.3xlarge Trainium2 instance.","Offline batch workloads processed 11–70% more audio per dollar versus previous compute options.","Rust data‑parallel runtime achieved a 7.5% throughput boost over the multiprocessing baseline."],"whyItMatters":"Shows that hardware‑specific tuning can substantially cut costs and raise throughput for large‑scale speech AI, encouraging wider use of AWS Trainium for voice services.","category":{"slug":"hardware","name":"Hardware & Compute","url":"https://digestai.news/category/hardware"},"entities":{"companies":["Deepgram","Amazon Web Services"],"models":["Nova-3"],"people":[]},"firstPublishedAt":"2026-10-19T16:23:22Z","updatedAt":"2026-10-19T16:23:22Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"AWS Machine Learning Blog","title":"How Deepgram optimized voice AI at scale with AWS Trainium","url":"https://aws.amazon.com/blogs/machine-learning/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium","publishedAt":"2026-10-19T16:23:22Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Deepgram improves Nova-3 speech-to-text efficiency on AWS trainium2\", 19 October 2026, https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2","publisher":"Digest AI","title":"Deepgram improves Nova-3 speech-to-text efficiency on AWS trainium2","datePublished":"2026-10-19T16:23:22Z","url":"https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}