ThanosStoreGatewayBucketHighOperationFailures#
This alert indicates that Thanos Store Gateway fails to perform read/write operations to S3 bucket.
Troubleshooting Hints#
kubectl get pods -n <monitoring_namespace> | grep store-gateway # check pod is up and running
kubectl logs <thanos_store-gateway> -n <monitoring_namespace>
kubectl get pods -n <thanos_store-gateway> -n <monitoring_namespace> -o yaml | grep bucket # check bucket name is correct
kubectl get svc -n <monitoring_namespace> | grep store-gateway # service should be present
Tip
Where <monitoring_namespace> can be:
- Zone Cloud:
monitoring,monitoring-clientormonitoring-app - Zone Infra:
monitoringormonitoring-consumption
- Notice the
namespacelabel in the alert
To check if bucket name is correct, follow bucket nomenclature documentation.