Aller au contenu

GrafanaDuplicatedTitleDashboards#

The alert is raised if two dashboards with the same title are loaded in the same Grafana instance (namespace).

Tip

Below, namespace can be:

  • monitoring
  • monitoring-client
  • Notice the namespace label in the alert.
  • monitoring
  • monitoring-app

Grafana loads dashboards from ConfigMaps if they have the label grafana-dashboard-caascad, grafana-dashboard-client or grafana_dashboard. One ConfigMap can define multiple dashboards.

Note

For Angel platform, the labels can be : grafana-dashboard-caascad, grafana-dashboard-app.

Troubleshooting hints#

Search for the ConfigMap#

Note the title and the namespace labels from the alert.

Open a terminal, connect to the zone specified in the obs_client label of the alert and search for the problematic dashboards:

TITLE=<title label in the alert>
NAMESPACE=<namespace label in the alert>

LABEL_GRAFANA_DASHBOARD="$([[ "$NAMESPACE" == "monitoring" ]] && echo 'grafana-dashboard-caascad' || echo 'grafana-dashboard-client')"

TITLE=<title label in the alert>
NAMESPACE=<namespace label in the alert>

LABEL_GRAFANA_DASHBOARD="$([[ "$NAMESPACE" == "dashboard-corp" ]] && echo 'grafana-dashboard' || echo "Not Supported yet")"

kubectl get ConfigMap -A -o json | \
  jq -r --arg TITLE "$TITLE" --arg LABEL_GRAFANA_DASHBOARD "$LABEL_GRAFANA_DASHBOARD" '
        .items[]
          | select(.metadata.labels[$LABEL_GRAFANA_DASHBOARD])
          | .metadata as $metadata
          | .data | to_entries[]
            | (.value | fromjson) as $value
            | select($value.title == $TITLE)
            | {
                key:        .key,
                name:       $metadata.name,
                namespace:  $metadata.namespace,
                uid:        $value.uid,
                title:      $value.title,
                labels:     $metadata.labels,
              }'

Solutions#

The duplicated dashboards belong to Caascad customers#

If the namespace where there is a duplicated dashboard is the namespace client-dashboards, please ask the support team to contact the client and explain to them how and why they need to modify the title of the dashboard in the MonitoringApp Gitea repository (reasons: the client will not be able to see one of the duplicated dashboards, and there is a risk of ContainerOutMemory on Grafana's pod if there are too many errors caused by this problem).

Note

This problem can only occur on Caascad zone.

Note: An improvement will be added to the GitOps Workflow to prevent the custumer from merging and deploying from their repository if there already is another dashoard with the same uid/title).

The duplicated dashboards belong to Caascad Teams#

Else, try to find which team is in charge of the namespace where the dashboard is and escalate the CAASINC and explain to them how and why they need to modify the title of the dashboard. They will need to fix it at the git level, and redeploy their dashboards.