AI Gateway vs Direct Provider: Cheapest Multi-Model Routing, Billing and Token Estimates

Short summary

A practical comparison of Vercel AI Gateway, OpenRouter, and direct provider SDKs for multi-model routing, focused on per-call cost reporting, token usage transparency, and request-shape compatibility. The author argues that cost-per-accepted-output is the real metric to optimize, and that a gateway earns its keep by removing hardcoded price constants from your codebase. Key evaluation criteria include per-call dollar visibility, OpenAI wire-format compatibility, pre-call price estimation, model metadata APIs, and support for non-chat workloads like embeddings and reranking.

• •Optimize cost per accepted output, not per-million-token price, since cheaper models may need retries that erase savings
• •Gateways should return per-call dollar amounts alongside completions so you can join cost with eval scores without a second pipeline
• •Evaluate routing layers on cost reporting granularity, request-shape compatibility, pre-call estimation, model metadata, and non-chat workload support

Generated with AI, which can make mistakes.