I tested the 'deterministic agent loop' claims with four experiments. They all failed — including my own fix.

Short summary

The author experimentally falsifies three core mechanisms used in production AI agents: lexical-overlap thresholds (44% accuracy), temperature-0 evaluators (70% consistency on open-ended text), and phase gates (unreliable task completion detection). Results challenge industry claims that deterministic constraints can reliably constrain LLM uncertainty at scale.

• •Lexical-overlap thresholds for task classification fail 50% of the time, especially on paraphrased and cross-lingual inputs
• •Temperature-0 outputs are only 70% consistent for open-ended text like evaluator reasoning fields, contradicting determinism claims
• •Phase gates don't reliably transform task completion into an objective fact, undermining the entire deterministic agent loop architecture

Generated with AI, which can make mistakes.