PrometheusOperatorSyncFailed

Prometheus operator pod creates the configuration and exposes it to Prometheus pods.

The prometheus operator will attempt to run the sync routine (reconcile) every time that a Prometheus object has been updated/created/destroyed.

Troubleshooting hints

• check prometheus-operator logs, it will probably provide more details regarding invalid configuration:

  kubectl logs <prometheus-operator_pod> -n <namespace>
  

  Where namespace can have the following values depending on zone type:

  CaascadNGOT

  • cloud: monitoring
  • client: caascad-monitoring

  • monitoring

• check custom resource Prometheus

 kubectl describe prometheus <prometheus_name> -n <namespace> | grep <invalid_configuration_parameter>

Where namespace can have the following values depending on zone type:

• cloud-caascad: monitoring
• cloud-client: monitoring-client
• cloud-app monitoring-app
• client: caascad-monitoring
• NGOT: monitoring

Note

The custom resource Prometheus manages:

 - Prometheus configuration
 - Prometheus statefulset (Prometheus pod)

The root cause can be:

• bad prometheus configuration