Keycloak 503 Errors#

Symptoms:#

Error in logs:
Unable to connect to host pocwatt-keycloak-cluster This indicates a disconnection from the database.

Known Cases:#

Unable to connect to host pocwatt-keycloak-cluster

Checks:#

Check the state of Keycloak pods:

kubectl -n keycloak get pod

Verify logs of the pods:

kubectl -n keycloak logs <pod_name>

Check Cluster Topology:

kubectl exec -it -n keycloak pocwatt-keycloak-cluster-0 -- patronictl list

You may see something similar to this:

...
pocwatt-keycloak-cluster-1 | 1XX.16.X.1XX | Replica | crashed |     |   unknown |
...

Action fix the issue:#

kubectl exec -it -n keycloak pocwatt-keycloak-cluster-0 -- patronictl failover

You might encounter something like this:

Warning

TypeError: Cannot mix str and non-str arguments xxxxxtimexxxxx - WARNING - Failing over to DCS xxxxxtimexxxxx Could not failover using Patroni API, falling back to DCS

Then use this code

kubectl delete -n keycloak pod pocwatt-keycloak-cluster-1

check the status:

kubectl exec -it -n keycloak pocwatt-keycloak-cluster-0 -- patronictl list

Reinitialize the pod:

kubectl exec -it -n keycloak pocwatt-keycloak-cluster-XX -- patronictl reinit pocwatt-keycloak-cluster pocwatt-keycloak-cluster-XX

Check one last time

kubectl exec -it -n keycloak pocwatt-keycloak-cluster-0 -- patronictl list