Connection pooling issue with external database in RHOCP

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.x
    • 3.x

Issue

  • A deployed microservice/application is not able to connect to external database.

[dd-mm-yyyy hh:mm:ss] [DASHBOARD-xxx connection adder] [com.xxx.xxx.xxx.xxx.xxx] {DASHBOARD-xxx - Added connection xxx.jdbc.driver.xxx}
[dd-mm-yyyy hh:mm:ss] [DASHBOARD-xxx connection closer] [com.xxx.xxx.xxx.xxx.xxx] {DASHBOARD-xxx - Closing connection xxx.jdbc.driver.xxx: (connection has passed maxLifetime)}

Resolution

  • The issue and the related error is not caused due to any functionality of RHOCP.

  • The error reported here is a cause due to External Database request limit or Underlying Network/Storage and this needs to be verified by the Networking Team or the Database vendor accordingly.

Root Cause

  • The microservice / application is not able to connect to the external database because it is exceeding the maxLifetime of its connection as the appropriate value of maxLifetime is not updated in the configuration of the concerned database or due to some connectivity issues in the underlying network.

Diagnostic Steps

  • Look for the health of the concerned microservice to confirm its in RUNNING state.
$ oc describe <microservice-pod-name> -n <project-name>
  • If the state of the microservice / application is RUNNING in the above step, then check for the service logs to find out the root cause which eventually signifies that the connection with the database is getting failed as it is exceeding the maxLifetime.
$ oc logs <microservice-pod-name> -n <project-name>

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments