Amazon SageMaker AI streams Kokoro-82M TTS per sentence to halve voice latency
The post explains how to stream text‑to‑speech for AI voice agents by sending each sentence to a hosted Kokoro-82M model on an Amazon SageMaker AI real‑time endpoint. The model runs on a single ml.g5.xlarge GPU instance, keeping latency and inference cost low. By using the endpoint’s bidirectional WebSocket stream, audio synthesis starts as soon as a sentence is generated, instead of waiting for…
Key points
- Kokoro-82M TTS runs on a SageMaker AI real‑time endpoint using an ml.g5.xlarge GPU instance.
- Streaming each sentence reduces time to first audio from 329.3 ms to 156.7 ms, a 52.4 percent improvement.
- Bidirectional streaming makes voice latency 2.4 × faster locally and 3.5 × faster in deployed AgentCore runtime versus batch synthesis.
Benchmarks show the streaming approach cuts the mean time to the first audio byte from 329.3 ms to 156.7 ms, a 52.4 percent relative improvement. In an end‑to‑end test with a generative agent, bidirectional mode is 2.4 times faster than batch synthesis on a local container and 3.5 times faster on the deployed AgentCore runtime. The guide also compares two transport options for delivering audio to browsers, noting that base64‑encoded audio over Server‑Sent Events adds roughly 33 percent payload overhead and may misalign MP3 frames, while a unified binary‑and‑JSON stream avoids those costs.
The story so far
5 episodes →- Amazon SageMaker AI streams Kokoro-82M TTS per sentence to halve voice latencythis story
Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI
AWS Machine Learning Blog · 22 October 2026
Loading the full article…
This text was published by AWS Machine Learning Blog and written by Joseph Cottingham. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
2sources- Stop guessing auto scaling targets for TTS on Amazon SageMaker AIPrimary source · AWS Machine Learning Blog ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- Amazon Quick is HIPAA‑eligible for AI workloads in healthcare · 1 src
- Microsoft says Storm-0940 used AI-driven ARTEX framework in attacks · 11 src
- AAIF adds four new agent projects from Anthropic, OpenAI, Alibaba · 2 src
- Phylo partners with Bayer to deploy Biomni Lab AI agents for drug research · 1 src
- Google CEO says Gemini agents will be integrated into Search through an agentic layer · 2 src
Comments
via GitHub Discussions