ThanosSidecarBucketOperationsFailed#
This alert indicates that Thanos Sidecar fails to perform read/write operations to S3 bucket.
Troubleshooting Hints#
kubectl get pods -n <monitoring_namespace> | grep prometheus # check 2 pods are up and running
kubectl logs <prometheus_pod> -c thanos-sidecar -n <monitoring_namespace> # check logs of sidecar container
kubectl get pods <prometheus_pod> -n <monitoring_namespace> -o yaml | grep bucket # check bucket name is correct
kubectl get svc -n <monitoring_namespace> | grep sidecar # service should be present
Tip
Where <monitoring_namespace> can be:
- Zone Cloud:
monitoring,monitoring-clientormonitoring-app - Zone Infra:
monitoringormonitoring-consumption
- Notice the
namespacelabel in the alert
To check if bucket name is correct, follow bucket nomenclature documentation.