I Added `--retries 2` to Our Pipeline. It Took Me Eight Months to Understand What I'd Done.

Short summary

An engineer recounts how adding --retries 2 to a flaky CI pipeline masked real bugs by reducing their detection rate from 30% to 2.7% per run. A payment double-submit bug that fired 5% of the time became effectively invisible (1 in 8,000 runs) but still affected 1 in 20 users, reaching 14 customer complaints before discovery. The core lesson: retries silence the alarm without fixing the fire, and users don't get retries.

• •Adding --retries 2 to CI masked real race conditions by reducing detection from 30% to 2.7% of runs
• •A 5% payment double-submit bug became invisible (1-in-8,000) but still hit 14 customers
• •Retries suppress signal without fixing root causes; users experience bugs the CI never reports

Generated with AI, which can make mistakes.