Logging#

Prerequisites#

Inputs#

export GIT_PATH="$HOME/git/"
export ZONE_NAME="ocb-..."
export DOMAIN_NAME="caascad.com" # ou cloudservicesfactory.com
export INFRA_ZONE_NAME="infra-..."
export BRANCH_NAME="pf-..."

Warning

During this procedure, several steps require restarts of different components in cloud zones. Before each restart:

check there is no CAASCHR that is planned on the same component at the same time
warn supervsion team on #supervision about the component/zone that is going to be restarted

envs-ng#

Tip

Many steps will consist of running trackbone apply -z "$ZONE_NAME" -c ...

Reusing the shell setup here is a good idea!

Repository: caascad/terraform/envs-ng

# git clone git@git.corp.caascad.com:caascad/terraform/envs-ng.git "$GIT_PATH/caascad/terraform/envs-ng"
cd "$GIT_PATH/caascad/terraform/envs-ng"
git pull
git switch -c "$BRANCH_NAME"
nix-shell

References#

Logging functional tests#

The functional tests have been deployed during the monitoring deployment.

Check on the CI (Staging or Production - change for the cloud zone of the zone being deployed) that the zone was added to the functional tests for logging.

Deploy Loki (cloud-caascad and cloud-client)#

Pre-verification in caascad-zones#

Note

The next actions are to be done for loki-cloud-caascad and loki-cloud-client.

Check the retention in zones.cue before deploying.

Section : monitoring/logs/caascad ; vérifier ces paramètres :
- retention : retention depends on what the client asks for. WARNING : it should correspond to the retention defined in the "lifecycle rule" of the S3 bucket (but that's not so easy to check this)

Prerequisistes#

In order to deploy, it is necessary to be authenticated to Vault:

export VAULT_ADDR=https://vault.${INFRA_ZONE_NAME}.caascad.com
vault login -method oidc

Loki (cloud-caascad and cloud-client)#

trackbone apply --non-interactive -z ${ZONE_NAME} -c loki-cloud-caascad
trackbone apply --non-interactive -z ${ZONE_NAME} -c loki-cloud-client

Promtail (cloud-caascad)#

Deployment#

for Caascad zones: cd contexts/caascad
for NgotAlpha zones: cd contexts/ngotalpha

trackbone apply --non-interactive -z ${ZONE_NAME} -c promtail

Verification logging stack#

Automated functional tests in Concourse#

Connect to Concourse

fly switch -z "${INFRA_ZONE_NAME}" -n test
fly -t "${INFRA_ZONE_NAME}.infra" login -n test

Launch automated functional tests :

for stack in logging; do
  fly -t ${INFRA_ZONE_NAME}.infra trigger-job -j "f-tests-${stack}-${ZONE_NAME}/${stack}-${ZONE_NAME}"
done

Check results in Concourse UI or with this :

for stack in logging; do
  fly -t ${INFRA_ZONE_NAME}.infra jobs -p "f-tests-${stack}-${ZONE_NAME}"
done

Automated functional tests with scripts only#

Run script verification for Loki

Run the script in application/caascad-loki/tests

kswitch "${ZONE_NAME}"
cd ${GIT_PATH}/caascad/applications/caascad-loki
git checkout master
git pull --rebase
(cd tests && ./check_loki_latest_logs.sh ${ZONE_NAME} logging && echo "OK check_loki_latest_logs.sh logging")

Non-automated tests#

Grafana-infra#

On https://grafana-infra.${ZONE_NAME}.caascad.com ensure that everything is working: check Loki/Operational dashboard.