# Deepgram improves Nova-3 speech-to-text efficiency on AWS trainium2

Digest AI · Hardware & Compute · published 2026-10-19T16:23:22Z

Canonical: https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2

## Summary

Deepgram processes quadrillions of audio tokens a year for more than 200,000 developers and 1,400 organizations. To cut inference costs, the company ported its Nova-3 speech‑to‑text model to AWS Trainium2 using TorchNeuron, replacing several layers with pre‑built kernels from the public NKI Library and tuning them with Neuron Explorer and the Neuron Agentic Development toolkit. The final setup runs eight model replicas on a single EC2 trn2.3xlarge instance.

In offline, throughput‑oriented batch workloads the Trainium2 configuration delivered 11–70 percent more audio per dollar than Deepgram’s prior compute options, based on instance pricing at the time of evaluation. A custom Rust data‑parallel runtime eliminated Python interpreter overhead and added a 7.5 percent throughput gain over a multiprocessing Python baseline. These improvements lower costs for Deepgram’s customers and demonstrate that AWS Trainium can be an efficient platform for large‑scale voice AI inference.

## Key points

- Deepgram’s Nova-3 runs eight replicas on a single trn2.3xlarge Trainium2 instance.
- Offline batch workloads processed 11–70% more audio per dollar versus previous compute options.
- Rust data‑parallel runtime achieved a 7.5% throughput boost over the multiprocessing baseline.

## Why it matters

Shows that hardware‑specific tuning can substantially cut costs and raise throughput for large‑scale speech AI, encouraging wider use of AWS Trainium for voice services.

## Sources

1. [How Deepgram optimized voice AI at scale with AWS Trainium](https://aws.amazon.com/blogs/machine-learning/how-deepgram-optimized-voice-ai-at-scale-with-aws-trainium) (AWS Machine Learning Blog, 2026-10-19, primary source)

## Cite

Digest AI, "Deepgram improves Nova-3 speech-to-text efficiency on AWS trainium2", 19 October 2026, https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/deepgram-improves-nova-3-speech-to-text-efficiency-on-aws-trainium2.json