Building Incident AI That Engineers Actually Trust

Short summary

Building an AI incident detection system requires strong foundations of system context—topology, deploy tracking, and incident history—before any automated reasoning can be trusted. The authors' first attempt achieved only 35% engineer agreement on top hypotheses because it lacked deploy awareness, but adding deploy tracking and a context graph raised agreement to 70%. The article outlines a six-layer architecture emphasizing normalization, context graphs, evidence retrieval, and focused reasoning over raw data summarization.

• •System context (topology, deploys, postmortems) must precede AI reasoning—without it, the system confidently blames the wrong service
• •Adding deploy awareness alone improved engineer agreement on top hypotheses from 35% to 70%
• •Six-layer architecture: normalization, context graph, evidence retrieval, reasoning, summarization, and feedback—each with clear contracts

Generated with AI, which can make mistakes.