Prompt Caching and Cost Control in Python

Short summary

Prompt caching reduces LLM costs by 90% for repeated system prompts and RAG context by reading cached prefixes at 0.1× input price instead of reprocessing. Use model routing to triage requests to cheaper models, and the Batches API for non-latency-sensitive work at 50% discount. Measure token usage across cache writes, reads, and uncached portions before optimizing.

• •Prompt caching reads repeated prefixes at 0.1× cost instead of reprocessing (writes cost 1.25–2× but break even in 2–3 requests)
• •Model routing sends hard requests to Opus, easy ones to Haiku; significantly reduces costs without quality loss
• •Batches API offers 50% cost savings for non-latency-sensitive work like report summarization or bulk classification

Generated with AI, which can make mistakes.