PrometheusAllRemoteStorageUnexpectedlyDropped

Some samples (aka metrics) are dropped without having been sent over Remote Write.

Because we are loosing data (the dropped samples) this alert is critical. Its criticity depends on how many data we are loosing.

This alert has few differences with PrometheusRemoteStorageUnexpectedlyDropped:

• PrometheusAllRemoteStorageUnexpectedlyDropped fires when we are loosing data when sending on All Remote-Write endpoints. For this reason, its severity is critical. Moreover, the labels remote_name and url are missing because all of them are impacted. This alert is pending for a short time before firing (for=5m).
• PrometheusRemoteStorageUnexpectedlyDropped fires for each Remote-Write endpoint where we are loosing data. Because other Remote-Write endpoints may have no problem, the severity is only warning. Most of the time, the problem repairs itself so the alert is pending for a long time before firing (for=30m).

Troubleshooting and solutions are the same as PrometheusRemoteStorageUnexpectedlyDropped.