I Made My Voice Agent Feel Faster by Streaming Sentences, Not Audio

Short summary

A developer reduced voice agent latency by streaming LLM text at sentence boundaries and overlapping TTS synthesis with audio playback, achieving ~5x faster time-to-first-byte. The approach uses a simple regex splitter, a ThreadPoolExecutor for parallel synthesis, and a playback queue — no custom audio protocol needed. Additional gains came from disabling extended model thinking and conditionally adding Google Search only when needed.

• •Stream TTS at sentence boundaries instead of waiting for full model response
• •Overlap synthesis and playback using a ThreadPoolExecutor and queue to eliminate inter-sentence gaps
• •Cut time-to-first-byte ~5x by disabling thinking budget and conditionally adding search tools

Generated with AI, which can make mistakes.