Detecting silent LLM cost spikes in AI agents using percentile-based token baselines

Short summary

Silent token consumption spikes in production AI agents can inflate LLM bills by 40% without any visible errors. The article proposes using percentile-based baselines (p50, p95, p99) to detect anomalous token usage patterns, distinguishing one-off complex queries from sustained runaway loops. A practical alerting strategy flags multiple consecutive runs above p95 with at least one above p99, enabling teams to catch retry loops and tool recursion before costs spiral.

• •Percentile-based baselines (p50/p95/p99) detect cost anomalies that raw token counts miss
• •Sustained spikes — multiple runs above p95 with one above p99 — signal systematic issues like retry loops
• •Tracking failed runs and retries is critical since they are where costs spike silently

Generated with AI, which can make mistakes.