Aller au contenu

LokiFailedToPushLogs#

Context#

This alert appears when Loki is not able to flush :

  • memory chunks in backend : S3 bucket (or blob for Azure)

Chunk#

A chunk that is in memory is sent to the backend (s3...) for the following different reasons:

  • the chunk is full: size defined in the conf
  • the chunk is no longer updated without update for a certain period, and when this period is reached, it sends (period defined in the conf)
  • the chunk is too old

Once one of the criteria becomes true, it is flushed indefinitely as long as the flush fails. And all this, every 30 seconds (flush_check_period parameter).

So, until the memory is full and the ingester pod doesn't restart, the flush is sent every 30 seconds.

Index#

An index table is flushed every minute.

Potential problems#

It appears in one of the following situations:

  • network problem
  • bad S3 link configured in Loki configmap
  • bad AK/SK
  • ...

Alert explanation#

See the details here.

Troubleshooting#

Check LokiFlushQueueNotEmpty alert documentation for more details.