Aller au contenu

ThanosQueryHighDNSFailures#

This alert indicates that Thanos Query is having high number of DNS failures.

This can be due tu different reasons:

  • DNS issue
  • Thanos endpoint not available

Troubleshooting Hints#

kubectl get endpoints -n <monitoring_namespace> | grep thanos  # compact, querier, sidecar, store-gateway should be present

Tip

Where <monitoring_namespace> can be:

  • Zone Cloud: monitoring, monitoring-client or monitoring-app
  • Zone Infra: monitoring or monitoring-consumption
  • Notice the namespace label in the alert

Tip

On NGOT, remember that there are 4 possibles queriers :

  • Querier client: namespace of the client. This is the querier connected to the Grafana of the client.
  • Querier Grafana: namespace grafana-obs-corp-prd/grafana-obs-corp-stg. This is the querier connected to the Grafana for NGOT teams. On the other side, it is connected to Thanos-Querier (proxy).
  • Querier Proxy: namespace monitoring-stack-corp-obs-corp-prd/monitoring-stack-corp-obs-corp-stg. This is the proxy (or dispatcher) to all the Thanos components
  • Querier Ruler: namespace monitoring-stack-corp-obs-corp-prd/monitoring-stack-corp-obs-corp-stg. This is a dedicated querier for Thanos Ruler.