Aller au contenu

Create logs for pod in Loki#

Two possible cases :

  • we want to retrieve a line of logs from a specific application and inject it into another application
  • we want to inject any line of logs into an application

For the first case, you can initialize the following variables : 

log_lines=""
source_pod_name=<source_pod_name>
destination_pod_name=<destination_pod_name>

For the second case :

  • go directly to step : Connect to the node where the destination pod is scheduled
  • you can initialize the following variables : 
    log_lines="<log_lines>"
destination_pod_name=<pod_name>

Connect to the node where the source pod is scheduled#

GET IP node#

kubectl get pod -n ${namespace} -o wide | grep $source_pod_name

Connect to the node#

ssh cloud@bst.<zone_name>.caascad.com
ssh cloud@<node-ip>

or

kubectl node-shell <node-name>

Identify the path of the logs directory#

sudo cat /var/run/promtail/positions.yaml | grep $source_pod_name

Example of result with ingester : 

/var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/0.log: "145"

Get log lines#

Example : 

tail -n 50 /var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/0.log

Retrieve the desired log lines and put them in the variable log_lines.

Connect to the node where the destination pod is scheduled#

GET IP node#

kubectl get pod -n ${namespace} -o wide | grep $destination_pod_name

Connect to the node#

ssh cloud@bst.<zone_name>.caascad.com
ssh cloud@<node-ip>

or

kubectl node-shell <node-name>

Identify the path of the logs directory#

sudo cat /var/run/promtail/positions.yaml | grep $destination_pod_name

Example of result with ingester : 

/var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/0.log: "145"

Create log line#

You can create a new file in the directory found by incrementing the number in the file name.

echo "$log_lines" | sudo tee -a <log_path>/1.log

Warning

  • the file must not exist
  • the timestamp of the log line must not be too old

Example with ingester : 

echo '{"log":"level=error ts=2022-04-15T16:12:37.335048982Z caller=flush.go:220 org_id=fake msg=\"failed to flush user\" err=\"RequestError: send request failed\\ncaused by: Put \\\"http://osstoto.eu-west-0.prod-cloud-ocb.orange-business.com/loki-cloud-client-ocb-test05/fake/fcbc108991757af8%3A17ffacacf83%3A17ffb01d13e%3A4808e0ce\\\": test\"\n","stream":"stderr","time":"2022-04-15T16:12:37.335186713Z"}' | sudo tee -a /var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/1.log

Result#

Promtail will read the previously written log line and send it to Loki.

You can find it by filtering on the filename label.

Example of LogQL request with ingester : 

{namespace="logging-client", container="ingester", filename="/var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/1.log"}

Cleanup#

Once the tests are finished, delete the log files previously created.

sudo rm /var/log/pods/logging-client_ingester-6685469d69-r56nw_76676e17-e766-4ceb-a2cb-5ac8c01bfbc3/ingester/1.log