A Verdict Is Not Evidence. Test Is Where I Learned the Difference.

Short summary

The author discovered that an AI agent's Review verdict (pass-with-risk) is a judgment, not proof—only actually running tests produces evidence. They built a verification matrix mapping every requirement to concrete evidence (command run, manual QA, or output comparison) and treat 'not run' as an owned risk rather than a silent pass. The key insight: a skipped check looks identical to a passing one on the page, so the matrix must require evidence for every row before allowing a ship recommendation.

• •Review verdicts are judgments about intent; only Test produces evidence of actual behavior
• •A verification matrix requires every requirement to have command-run, manual-QA, or output-comparison evidence—'not run' is an owned risk, not a silent pass
• •Keeping records of fixed failures matters: the matrix is proof the check ran, not just a snapshot of final state

Generated with AI, which can make mistakes.