Every AI provider fails in its own way. I stopped checking status codes and built an error model instead.

Short summary

The author built an API gateway routing between OpenAI, Anthropic, and Gemini, and found that error handling—not integration—was the real challenge. Each provider returns errors in different shapes with different meanings for the same HTTP status codes, making naive status-code forwarding insufficient. The solution: normalize all provider errors into a fixed internal error model with categories like rate_limited, overloaded, and quota_exceeded, enabling provider-agnostic retry decisions.

• •Different LLM providers return errors in incompatible formats even for the same HTTP status codes
• •Normalizing errors into a fixed error_class set (rate_limited, overloaded, quota_exceeded, etc.) decouples retry logic from provider specifics
• •Raw provider errors are logged for debugging while callers receive a consistent, categorized error response

Generated with AI, which can make mistakes.