Log retention#

There are two types of retention:

for logs in s3
for index in s3 with boltdb-shipper

In theory, the two values can be different.

In reality, if the log retention time is shorter than the index retention time, we will still be able to see related labels but will be unable to retrieve the deleted log content.

And if the index retention time is shorter than the log retention time, we will not be able to access the content of the logs for which the indexes have been deleted.

Note

In Caascad current implementation, the two retentions values are equal.

Log retention in s3#

The log retention in s3 needs top be configured/updated in two places:

FE console#

In Object Storage service :

click on the s3 bucket where the logs are stored. For more information, see bucket_nomenclature.
click on Lifecycle Rules menu
find the rule where expiration is defined
click on edit button
change Delete After field

Caascad-zones#

The log retention in s3 needs to be modified in caascad-zones.

Note

It is possible to specify different retentions for different tenants in loki, by creating several rules with different object name prefixes. It works because in loki there is a directory to create for each tenant.

Index retention in s3 with boltdb-shipper#

It's the same retention as for the logs, and the information is also in git repo caascad-zones.

Note

The retention period must be multiple of the table period. See index_schema_config for more information.

The effective retention, is the retention and one period.

Example with a retention period of 7 days and a table period of 7 days:

After modification of the index retention, check the log of table-manager :

if retention has increased, the number of tables should increase.
if retention has decreased, the number of tables should decrease.