Confront, Don't Assert

Short summary

AI code auditors are tuned to minimize false positives by filtering on confidence—but confidence isn't a proxy for truth. Research shows they miss real bugs 40+ percentage points more often than they invent false ones, especially when documentation looks right but code doesn't match it. Auditors should surface findings with explicit conditions for falsification, prioritizing accuracy over developer convenience.

• •Confidence filtering optimizes for developer convenience, not detection accuracy
• •LLM judges miss real bugs 40+ points more often than they create false positives
• •Surface findings with falsifiability conditions, not confidence thresholds

Generated with AI, which can make mistakes.