Service Account K8S & Approle securisation GitOps (DT Store)#

Objectives#

In order to deploy resources on the kubernetes cluster with GitlabCI, a service account was created in the needed namespaces. This service account has restricted roles. To use it, you'll need a kubeconfig file stored in vault. To get it, you can follow the documentation below.

Technical details#

The K8S Service Account and associated Vault approle/role and policies are handled by the configuration : https://git.corp.caascad.com/caascad/terraform/envs-ng/-/blob/master/contexts/ngot/k8s_service_account.cue

The configuration will provision for different type of service zones described in the envs-ng configuration K8S token that will be held into Vault. For each of these K8S Token a vault approle/role will be created. This will then allow a separated access for each K8S Service Account Token.

Each K8S Service Account is bound to specific target namespace(s) and allows only some verbs (API) to specified resources.

External documentation#

K8S Service Account : https://kubernetes.io/docs/concepts/security/service-accounts/

Vault Approle Auth Method : https://developer.hashicorp.com/vault/docs/auth/approle

Steps#

Authenticate to vault using a specific appRole
Get JWT token from vault
Configure kubectl (choose one)
- Build the kubeconfig file using the JWT token for the user auth
- OR by using the --token option in kubectl
Deploy ressources to k8s

How to retrieve Vault Approle Token#

To Authenticate vault and get the K8S account token, you need to query Vault API to generate a token associated to the Approle

First step requires you to retrieve the role ID by supplying its name to vault
```
vault read auth/approle/role/<role_name>/role-id
```
Second step is using this role ID to retrieve the secret ID:
```
vault write -f auth/approle/role/<role_name>/secret-id
```
Once both role-id and secret-id obtain you can generate a Vault Token that is bound to the specific role with the following command :

vault write auth/approle/login \
  role_id=<role_id> \
  secret_id=<secret_id>

Examples#

K8S Service Account and bound role :#

K8S Service Account definition

$ kubectl get sa svc-monitoring-stack-client-test06 -n monitoring-stack-client-obs-test06 -o yaml
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  creationTimestamp: "2024-03-06T09:28:54Z"
  name: svc-monitoring-stack-client-test06
  namespace: monitoring-stack-client-obs-test06
  resourceVersion: "143671916"
  uid: e09bdd70-7db5-4b45-8dd9-7832601c412a
secrets:
- name: svc-monitoring-stack-client-test06-token-<xxxxx>

K8S role :

$ kubectl describe role deploy-prometheus-rules-and-probes -n monitoring-stack-client-obs-test06
Name:         deploy-prometheus-rules-and-probes
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources                              Non-Resource URLs  Resource Names  Verbs
  ---------                              -----------------  --------------  -----
  probes.monitoring.coreos.com           []                 []              [list watch create update patch get delete]
  prometheusrules.monitoring.coreos.com  []                 []              [list watch create update patch get delete]

Vault Token#

Get the role_id

$ vault read auth/approle/role/k8s-sa/role-id
Key        Value
---        -----
role_id    06d71d8d-8b0e-f636-ffd6-4892e7b28191

Get the secret_id

$ vault write -f auth/approle/role/k8s-sa/secret-id
Key                   Value
---                   -----
secret_id             f9b4a906-fa35-7dea-3353-e94a8f35c041
secret_id_accessor    73935313-d1c7-91a9-f77e-8e624db44496
secret_id_ttl         0s

Login with role_id and secret_id

$ vault write auth/approle/login role_id=06d71d8d-8b0e-f636-ffd6-4892e7b28191 secret_id=f9b4a906-fa35-7dea-3353-e94a8f35c041
Key                     Value
---                     -----
token                   hvs.CAESIPA6__8aUFv_oTml4EH3EUehP48-6ZXQUY3TIyvl2w1gGh4KHGh2cy5QMWt1RFd3ZjRITUxYejdtZzJPNDRkS1E
token_accessor          7LCYCWWGvWE2hexHir5JEwmb
token_duration          768h
token_renewable         true
token_policies          ["default" "read-k8s-sa-token"]
identity_policies       []
policies                ["default" "read-k8s-sa-token"]
token_meta_role_name    k8s-sa

Login with token

$ vault login -method token
Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                     Value
---                     -----
token                   hvs.CAESIPA6__8aUFv_oTml4EH3EUehP48-6ZXQUY3TIyvl2w1gGh4KHGh2cy5QMWt1RFd3ZjRITUxYejdtZzJPNDRkS1E
token_accessor          7LCYCWWGvWE2hexHir5JEwmb
token_duration          767h59m39s
token_renewable         true
token_policies          ["default" "read-k8s-sa-token"]
identity_policies       []
policies                ["default" "read-k8s-sa-token"]
token_meta_role_name    k8s-sa

$ vault read secret/zones/fe/obs-test06/svc-grafana-client-test06/sa_token
Key                 Value
---                 -----
refresh_interval    768h
kubeconfig_data     map[token:eyJhbGciOiJSUzI1NiIsImtpZCI6ImNZY3NiMzFld1V6V1JfMlZjS0o5TXlSRG5yS3hNMm5TSzJvaTdGOWpUb1kifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJncmFmYW5hLWNsaWVudC1vYnMtdGVzdDAyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InN2Yy1ncmFmYW5hLWNsaWVudC10ZXN0MDItdG9rZW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoic3ZjLWdyYWZhbmEtY2xpZW50LXRlc3QwMiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjNhMzJmNzA0LTRlZTctNDYzZi1hZjdjLWUwOGQ2ODQzOWE1YSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpncmFmYW5hLWNsaWVudC1vYnMtdGVzdDAyOnN2Yy1ncmFmYW5hLWNsaWVudC10ZXN0MDIifQ.OFdWq6cnob84VKF6uqM4D7VLZk2HX09CBbxmq7K03m6XhzRjMSuyHoo2xh8O-XzqnEPXOlaeVsG-0INb4S4PxAJIWHe7-2cuJpojZlANqIV3FiNPhYGL43oISxLXjt2NRA7e01suJ6yKJMUyRQXwhkp2Pdza5kwKf6ww0kabekfmmXLix1wc4vf6HvLWc8MungcfibkT8vX5QzPM9yU8yL1bI5Au9l9KeZTncNDfGuskUCLa8LLhjizfbYlKi7FF6N0CSIPUELCf1Sbxkwkl-snXDnv5c0h4hA8cV3ZktluSm91FxbeDH6eWWsMY7ZVXDkwGcyy_J9UmuyIyqqFvtQ]