When the Free Executor Cost More: 40 Trials on Opus + Local Qwen Ended Up the Most Expensive Cloud Arm

Short summary

Empirical study of 40 trials shows pairing Opus with a locally-hosted Qwen executor—expected to be cheapest due to zero token cost—actually became the most expensive option. The culprit: Opus repeatedly re-reads Qwen's summaries through cached prompts, consuming 1.4–5.3× more tokens than Opus alone. Among cloud-only models, Opus+Haiku offers the best cost-reliability tradeoff.

• •Local Qwen executor costs zero tokens but forces Opus orchestrator to re-read summaries repeatedly via prompt cache, increasing overall cost 1.4–5.3×
• •Opus+Haiku cloud-only pairing provides the best balance: reliable success rate at mid-range cost
• •Haiku solo is cheapest but fails 25% of the time; Opus solo is reliable but costly

Generated with AI, which can make mistakes.