Back to feed
Dev.to
Dev.to
7/20/2026
Fix Silent Loki SSH Brute-Force Detection Gaps in Grafana

Fix Silent Loki SSH Brute-Force Detection Gaps in Grafana

Short summary

A Grafana dashboard showed clear SSH brute-force patterns but zero alerts fired due to three independent pipeline gaps: regex rules only matched one sshd log variant, Promtail never extracted source_ip as a field forcing expensive full-text queries, and Alertmanager route labels didn't match the alerting rule labels. The fix involves writing LogQL queries that match all failed-password variants, extracting IPs via regex parsers, and verifying Alertmanager label routing end-to-end.

  • sshd log format variants cause regex rules to silently miss valid-user brute-force attempts
  • Missing source_ip field in Promtail forces expensive line-filter queries and ruler timeouts
  • Alertmanager silently drops alerts when rule labels don't match any route.match tree entry

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more