A Reproducible Result Can Still Be a Lie

Short summary

Reproducibility in AI agent pipelines proves the recipe was followed on given inputs, but says nothing about whether those inputs were genuine — a doctored sanctions list reproduces a false "no match" perfectly forever. The authenticity gap can only be closed by source attestation (the origin signs its data) or quorum (multiple independent captures must agree), and artifacts must honestly label which guarantee they provide. Shipping a weak guarantee dressed in a strong one's cryptography is the most common and dangerous failure mode.

• •Reproducibility verifies computation, not provenance — a tampered input reproduces a false result deterministically
• •Two fixes exist: source attestation (origin signs data) and quorum (independent captures must agree), but most sources don't sign yet
• •Artifacts must label their guarantee level honestly — attested-by-source and attested-by-agreement are not equivalent

Generated with AI, which can make mistakes.