Nine Red Tests and No Way to Tell Which Ones Matter

Short summary

An SDET recounts how nine red test failures turned out to span four distinct categories—product bugs, test bugs, infrastructure issues, and unavailable dependencies—each with different owners and urgencies, yet all rendered identically in reports. The author argues that current tooling lacks the vocabulary to distinguish a blocked test from a genuine failure, making triage a guessing game. The piece calls for reports that encode failure category so engineers can prioritize effectively instead of working through failures alphabetically.

• •Nine test failures mapped to four distinct root-cause categories with different owners and urgency levels
• •Current CI reports render all failures identically, providing no signal for prioritization
• •Blocked tests carry zero product information but look identical to real regressions

Generated with AI, which can make mistakes.