Making LLM security verdicts verifiable: the evidence gate pattern

Short summary

USAP enforces that every AI security verdict must cite resolvable evidence (MCP sources, CVEs, repos, or internal standards), eliminating hand-wavy output by forcing models to satisfy a strict contract. The system evaluates against real incidents (Log4Shell, xz, Capital One, MOVEit) and gracefully degrades when evidence can't be found, marking uncertain axes as UNKNOWN rather than inventing numbers.

• •Evidence gate pattern: every security verdict must cite a resolvable source—MCP calls, CVEs, S3 artifacts, or in-repo standards
• •Eliminates hand-wavy output by enforcing a strict contract that blocks verdicts without real evidence
• •Evaluated against public incidents with precision/recall metrics; degrades gracefully instead of fabricating confidence scores

Generated with AI, which can make mistakes.