Aller au contenu

ThanosRuleQueueIsDroppingAlerts#

This alert is trigerred when Thanos Ruler is failing to queue alerts.

Troubleshooting Hints#

Note

  • NAMESPACE=monitoring-stack-corp-obs-corp-stg
  • NAMESPACE=monitoring-stack-corp-obs-corp-prd
  • Check if another alert is firing and may give you more information about the problem.
  • Check the logs of the Ruler.
  • Check if the connection to Alertmanager is working as usual.

Thanos Ruler errors#

Thanos ruler is connected to two Alertmanager instances.

Check Thanos Ruler logs to see the type of error and identify which alertmanager is the problem on:

kubectl get pods -n ${NAMESPACE}
kubectl logs -n ${NAMESPACE} -l app.kubernetes.io/name=thanos-ruler

Alertmanager errors#

Check Alertmanager logs and see errors:

kubectl logs -n ${NAMESPACE} <alertmanager-0|alertmanager-1>