Dev.to
7/20/2026

Fix Silent Loki SSH Brute-Force Detection Gaps in Grafana
Short summary
A Grafana dashboard showed clear SSH brute-force patterns but zero alerts fired due to three independent pipeline gaps: regex rules only matched one sshd log variant, Promtail never extracted source_ip as a field forcing expensive full-text queries, and Alertmanager route labels didn't match the alerting rule labels. The fix involves writing LogQL queries that match all failed-password variants, extracting IPs via regex parsers, and verifying Alertmanager label routing end-to-end.
- •sshd log format variants cause regex rules to silently miss valid-user brute-force attempts
- •Missing source_ip field in Promtail forces expensive line-filter queries and ruler timeouts
- •Alertmanager silently drops alerts when rule labels don't match any route.match tree entry
Generated with AI, which can make mistakes.
Is this a good recommendation for you?



