Cannot see logs in console and oc logs, oc exec, etc give tls internal server error
Issue
- We are getting "An error occurred while retrieving the requested logs." when trying to view logs for any pod in OCP 4.1 web interface.
WebSocket connection to 'wss://console-openshift-console.apps.example.com/api/kubernetes/api/v1/namespaces/openshift-console/pods/console-79b6c7bb87-gt2ck/log?container=console&follow=true&tailLines=1000&x-csrf-token=ESx4l2bhkAyUQ8nx9f0%2FmA3qThlJEI6IOptYX2N%2FSPBDwcQuQ1K91DDjT0I3J99QYF4rogNwgleVtq6FV%2BkL7Q%3D%3D' failed: Error during WebSocket handshake: Unexpected response code: 500
- The command line logs, exec, and rsh tools give a remote error
$ oc logs console-79b6c7bb87-gt2ck
Error from server: Get https://master0.example.com:10250/containerLogs/openshift-console/console-79b6c7bb87-gt2ck/console: remote error: tls: internal error
-
We have pending CSRs in an OpenShift 4 cluster after install
-
The attempt to
oc exec ...
is failing
$ oc exec marketplace-operator-768b99959-9pftm -n openshift-marketplace -- echo foo
Error from server: error dialing backend: remote error: tls: internal error
$ oc logs marketplace-operator-768b99959-9pftm -n openshift-marketplace
Error from server: Get https://master:10250/containerLogs/openshift-marketplace/marketplace-operator-768b99959-9pftm/marketplace-operator: remote error: tls: internal error
- kube-apiserver container has errors
$ sudo crictl ps | grep kube-api
239ec13eeaf4e beaf65fce4dc16947c5bd5d1ca7e16313234c393e8ca1c4251ac9b85094972bb About an hour ago Running kube-apiserver-operator 3 bd197ceb6f882
6f2bdcab072ca beaf65fce4dc16947c5bd5d1ca7e16313234c393e8ca1c4251ac9b85094972bb About an hour ago Running kube-apiserver-cert-syncer-8 1 6938a6ebc2c3d
e6b9db2994d07 0d8dcfc307048a0f0400e644fcd1c9929018103b15d0f9b23b4841f1e71937bc About an hour ago Running kube-apiserver-8 1 6938a6ebc2c3d
$ sudo crictl logs e6b9db2994d07
...
E0725 17:38:54.707552 1 status.go:64] apiserver received an error that is not an metav1.Status: &url.Error{Op:"Get", URL:"https://master:10250/containerLogs/openshift-kube-apiserver/kube-apiserver-master/kube-apiserver-8", Err:(*net.OpError)(0xc01ec89270)}
...
Environment
- Red Hat OpenShift Container Platform
- 4.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.