x509: certificate signed by unknown authority error in Red Hat OpenShift Monitoring components, after redeploying new certificates

Issue

• After running redeploy-certificates.yml playbook monitoring components have started to fail and show errors about invalid certificates in their logs (similar to below).

  $ oc logs -n openshift-monitoring grafana-xxx -c grafana-proxy
  ...
  server.go:2923: http: TLS handshake error from 10.47.4.1:40868: remote error: tls: unknown certificate authority
  ...
  $ oc logs -n openshift-monitoring alertmanager-main-0 -c alertmanager
  ...
  server.go:2923: http: TLS handshake error from 10.47.8.50:36282: remote error: tls: bad certificate
  server.go:2923: http: TLS handshake error from 10.47.4.1:57474: remote error: tls: unknown certificate authority
  ...
  $oc logs -n openshift-monitoring prometheus-k8s-0 -c prometheus
  ...
  level=error ts=2018-11-09T15:27:01.075454778Z caller=notifier.go:473 component=notifier alertmanager=https://10.47.4.81:9094/api/v1/alerts count=0 msg="Error sending alert" err="Post https://10.47.4.81:9094/api/v1/alerts: x509: certificate signed by unknown authority"
  

• Routes in the openshift-monitoring namespace (such as grafana) fail to resolve and return a 503 "Application is not available" error