{"version":1,"type":"story","url":"https://digestai.news/story/aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker","json":"https://digestai.news/story/aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker.json","markdown":"https://digestai.news/story/aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker.md","slug":"aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker","headline":"Amazon SageMaker AI streams Kokoro-82M TTS per sentence to halve voice latency","summary":"The post explains how to stream text‑to‑speech for AI voice agents by sending each sentence to a hosted Kokoro-82M model on an Amazon SageMaker AI real‑time endpoint. The model runs on a single ml.g5.xlarge GPU instance, keeping latency and inference cost low. By using the endpoint’s bidirectional WebSocket stream, audio synthesis starts as soon as a sentence is generated, instead of waiting for the full response.\n\nBenchmarks show the streaming approach cuts the mean time to the first audio byte from 329.3 ms to 156.7 ms, a 52.4 percent relative improvement. In an end‑to‑end test with a generative agent, bidirectional mode is 2.4 times faster than batch synthesis on a local container and 3.5 times faster on the deployed AgentCore runtime. The guide also compares two transport options for delivering audio to browsers, noting that base64‑encoded audio over Server‑Sent Events adds roughly 33 percent payload overhead and may misalign MP3 frames, while a unified binary‑and‑JSON stream avoids those costs.","keyPoints":["Kokoro-82M TTS runs on a SageMaker AI real‑time endpoint using an ml.g5.xlarge GPU instance.","Streaming each sentence reduces time to first audio from 329.3 ms to 156.7 ms, a 52.4 percent improvement.","Bidirectional streaming makes voice latency 2.4 × faster locally and 3.5 × faster in deployed AgentCore runtime versus batch synthesis."],"whyItMatters":"Lowering voice latency lets AI agents deliver audible responses in near real time, improving user experience for voice‑first apps and reducing compute costs.","category":{"slug":"agents","name":"Agents & Tools","url":"https://digestai.news/category/agents"},"entities":{"companies":["Amazon","AWS","Amazon SageMaker AI","Amazon Bedrock","Amazon Bedrock AgentCore","Amazon ECR"],"models":["Kokoro-82M"],"people":[]},"firstPublishedAt":"2026-10-22T16:30:06Z","updatedAt":"2026-10-22T17:45:58Z","sourceCount":2,"hasPrimarySource":true,"sources":[{"outlet":"AWS Machine Learning Blog","title":"Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI","url":"https://aws.amazon.com/blogs/machine-learning/stream-low-latency-speech-from-ai-agents-using-kokoro-tts-on-amazon-sagemaker-ai","publishedAt":"2026-10-22T17:45:58Z","type":"primary","primary":true,"lead":true},{"outlet":"AWS Machine Learning Blog","title":"Stop guessing auto scaling targets for TTS on Amazon SageMaker AI","url":"https://aws.amazon.com/blogs/machine-learning/stop-guessing-auto-scaling-targets-for-tts-on-amazon-sagemaker-ai","publishedAt":"2026-10-22T16:30:06Z","type":"primary","primary":true,"lead":false}],"sourceNotes":null,"discussions":[],"thread":{"title":"Accelerating Large Language Models Performance Chronicle","url":"https://digestai.news/thread/rbs-attention-provides-radius-bounded-sparse-prefill-for-long-context-llms","storyCount":5},"cite":{"text":"Digest AI, \"Amazon SageMaker AI streams Kokoro-82M TTS per sentence to halve voice latency\", 22 October 2026, https://digestai.news/story/aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker","publisher":"Digest AI","title":"Amazon SageMaker AI streams Kokoro-82M TTS per sentence to halve voice latency","datePublished":"2026-10-22T16:30:06Z","url":"https://digestai.news/story/aws-blog-shows-measured-auto-scaling-for-kokoro-82m-tts-on-sagemaker"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}