Voice agent's two-second silence makes callers talk over greeting

Short summary

A voice agent's two-second opening-turn silence caused callers to talk over the greeting because they assumed the line dropped. The author instrumented every pipeline stage—ASR (300ms), LLM (1100ms), TTS (480ms), network (120ms)—and eliminated avoidable waits by streaming LLM tokens to TTS at sentence boundaries and warming the TTS connection. A pre-synthesized filler acknowledgement covers the irreducible remaining latency, cutting perceived time-to-first-audio from ~2s to under 500ms.

• •Diagnosed 2s voice-agent latency by timestamping each pipeline stage: ASR 300ms, LLM 1100ms, TTS 480ms, network 120ms
• •Streamed LLM tokens to TTS at sentence boundaries, dropping LLM-to-first-audio from 1100ms to 250ms
• •Warmed TTS connection and added pre-synthesized filler acknowledgements to mask irreducible latency

Generated with AI, which can make mistakes.