Why I don't use an LLM to secure my LLM

Short summary

The author argues against using an LLM-as-judge on every request for AI security, citing latency, cost, non-determinism, and the fact that the security layer itself becomes jailbreakable. Instead, they propose a four-tier escalation model: regex for known patterns, classical ML for deterministic detection, optional transformers for deeper analysis, and LLM judgment only for genuinely ambiguous cases. This approach runs mostly deterministically and can operate air-gapped in your own VPC.

• •LLM-as-judge on every request adds hundreds of ms latency, per-token costs, non-deterministic verdicts, and is itself jailbreakable
• •Tiered detection (regex → classical ML → transformer → LLM) pays LLM prices only on the tiny fraction needing judgment calls
• •Deterministic tiers can run air-gapped in your VPC with zero calls to hosted models

Generated with AI, which can make mistakes.