Liveness Without Health Is Theater

Short summary

Liveness checks prove a scheduled job ran, but not that it succeeded—creating a blind spot where jobs run forever while silently failing. The fix is a two-marker protocol: a beat file written on every run and an ok file written only on success. A sweep reads both ages to classify jobs as STOPPED, FAILING, or HEALTHY, catching the dangerous middle state where the process is alive but the work is broken.

• •Liveness proves a job ran; health proves it succeeded—conflating them hides silent failures
• •Two-marker protocol: beat file on every run, ok file only on success
• •Sweep classifies jobs as STOPPED, FAILING, or HEALTHY based on both marker ages
• •Alert path canary ensures notifications can actually get out

Generated with AI, which can make mistakes.