Building Low-Latency Voice Agents with OpenAI’s GPT-Realtime-2.1 and GPT-Realtime-2.1-mini

Short summary

OpenAI's GPT-Realtime-2.1 and 2.1-mini models enable single-model voice agents that handle audio understanding and generation over a live connection, eliminating the STT-to-TTS pipeline latency penalty. The mini model adds reasoning and tool use at the same price as its predecessor, with configurable reasoning effort (minimal to xhigh) and p95 latency improved by 25% via better caching. Builders should pick the mini for high-volume everyday tasks and the full model for high-stakes or complex tool orchestration, while estimating per-session costs against real traffic patterns rather than raw token rates.

• •Single-model realtime API removes STT+TTS chaining, cutting end-to-end latency and improving conversational feel
• •gpt-realtime-2.1-mini adds reasoning and tool use at earlier mini pricing; full model offers stronger quality for complex orchestration
• •Caching cuts p95 latency 25%+ and discounts cached input tokens; use WebRTC for browser, WebSockets for server, SIP for telephony with ephemeral client secrets

Generated with AI, which can make mistakes.