Log index schema configuration

Schema configuration

In loki, tables for indexes are created periodically.

Each table is associated with a schema configuration.

In a schema configuration, we can define multiple options:

• period
• prefix of the tables
• schema version
• storage backend

To change these and not lose data in s3 with boltdb-shipper, you must add a new schema configuration.

The schemas configuration are in loki.cue.

Default values:

lokiDefaultExtraSchemaConfigOptions: {
    defaultAWSExtraSchemaConfig: [
        {
            from: "2021-05-27"
            index: {
                period: "24h"
                prefix: "loki_index_from_20210527_"
            }
            object_store: "aws"
            schema:       "v11"
            store:        "boltdb-shipper"
        },
    ]
    defaultAzureExtraSchemaConfig: [
        {
            from: "2021-05-27"
            index: {
                period: "24h"
                prefix: "loki_index_from_20210527_"
            }
            object_store: "azure"
            schema:       "v11"
            store:        "boltdb-shipper"
        },
    ]
}

Add a new schema configuration

In order to test new schema configuration, it is possible to override default values for specific environements.

To deploy new schema configuration on all environements, it needs to be placed in the default configuration.

Override example for testing purpose:

envs: ["ocb-test06"]: configurations: "loki-cloud-client": {
    helm: values: loki: ingester: extraSchemaConfig:
    [
        {
            from: "2021-11-21"
            index: {
                period: "48h"
                prefix: "loki_index_from_20211121_"
            }
            object_store: "aws"
            schema:       "v11"
            store:        "boltdb-shipper"
        },
        {
            from: "2021-12-23"
            index: {
                period: "24h"
                prefix: "loki_index_from_20211223_"
            }
            object_store: "aws"
            schema:       "v11"
            store:        "boltdb-shipper"
        },
    ]
}

In this example, the tables created between November 21 and December 23 will have a period of 48 hours and a prefix loki_index_from_20211121_ and will be store in s3 with boltdb-shipper.

And the tables created after December 23 will have a 24 hours period and a prefix loki_index_from_20211223_ and will also be stored in s3 with boltdb-shipper.

Importants notes :

• The day after the current date is specified in the value of from and prefix. Changes will not be visible until the next day.

• The table period must be a multiple of 24h.

• Don't delete the old schemas config.

Important

It's mandatory to put the day after the current date and don't delete the old schemas config, otherwise old indexes will be deleted.

Clean-up

Tables from old config schemas are never deleted even if the retention is exceeded. It's a bug in cortex, project used by loki.

For the moment to clean up we can set the period to 0 for config schemas whose retention is exceeded: there will be only one table.

This should be done only if all tables in the config schema have expired retention.

Bug

The related issue.

Verifications

To do the day after the current date:

• check log of table-manager : verify creation and deletion of tables and that there are no errors.

• verify tables in boltdb-shipper:

  s3cmd ls -r s3://loki-cloud-caascad-ocb-test05/index