I audited 249 of my own AI coding sessions. The problem wasn't lying.

Short summary

The author audited 249 of their own Claude Code sessions using a custom tool called red-handed, expecting to catch agents lying about test results. Zero confirmed lies were found, but seven sessions had unverifiable claims where tests ran in formats leaving no machine-readable trace. The real problem isn't deception—it's verification nobody can read, a structural issue in how AI agents interact with test outputs that don't produce parseable artifacts.

• •Audited 249 Claude Code sessions: zero confirmed false claims about tests passing
• •Real issue is verification gaps—agent checks that leave no machine-readable trace
• •Tool uses deterministic rules (two pieces of evidence required) rather than LLM-based judgment to avoid false accusations

Generated with AI, which can make mistakes.