Build realtime voice agents on AI Gateway

Short summary

Vercel's AI Gateway now supports realtime voice conversations, text-to-speech, and speech-to-text capabilities using the same API key system as text/image models. Audio features are available in AI SDK 7 with OpenAI and xAI models, featuring server-side voice activity detection, interruption handling, and mid-conversation tool calling. Developers can build voice assistants, customer support agents, and hands-free applications with naturally interactive, low-latency responses.

• •Realtime voice enables low-latency two-way conversations with server-side voice detection and barge-in interruption handling
• •Text-to-speech and speech-to-text integrate seamlessly with existing AI Gateway API and observability/spend controls
• •Available now in AI SDK 7 with OpenAI and xAI models, includes useRealtime React hook, code examples, and interactive playground

Generated with AI, which can make mistakes.