Building Maestro AI: Routing LLM Calls So Your Agent Doesn't Burn Sonnet on Summaries

Short summary

The author built Maestro AI, a harness-agnostic model router for Cursor and Claude Code that dispatches subtasks to the cheapest capable model — local Llama for summaries, hosted Qwen for medium refactors, Claude Sonnet for architecture. Key challenges included false-positive routing from ambiguous keywords like 'design,' agents sending meta-prompts instead of literal tasks, and zombie LiteLLM processes causing silent failures. The system adds per-call routing, session budgets, automatic escalation on quality-check failures, and tier-based fallbacks.

• •Maestro AI routes LLM subtasks by complexity: local models for easy tasks, premium models for architecture and multi-step refactors
• •Deterministic classification rules with bounded patterns prevent false-positive routing to expensive models
• •Post-call quality checks (non-empty, no refusal, valid JSON) trigger retry-then-escalate on failure

Generated with AI, which can make mistakes.