openshift-apiserver degraded and kube-apiserver not available in OCP 4
Issue
-
The
openshift-apiserver
Cluster Operator is degraded or not available:$ oc get co openshift-apiserver NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE openshift-apiserver 4.5.24 False False False 1h11m
-
There are "
panic
" errors in thekube-apiserver
pods:2021-01-01T00:00:00.000000000Z E0101 00:00:00.000000 18 runtime.go:78] Observed a panic: &errors.errorString{s:"killing connection/stream because serving request timed out and response had been started"} (killing connection/stream because serving request timed out and response had been started)
-
The
status
of theopenshift-apiserver
Cluster Operator has the followingmessage
:- lastTransitionTime: '2021-01-29T02:27:11Z' message: 'APIServicesAvailable: "apps.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/apps.openshift.io/v1?timeout=32s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "authorization.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/authorization.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "build.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/build.openshift.io/v1?timeout=32s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "image.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/image.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "oauth.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/oauth.openshift.io/v1?timeout=32s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "project.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/project.openshift.io/v1?timeout=32s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "quota.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/quota.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "route.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/route.openshift.io/v1?timeout=32s: context deadline exceeded) APIServicesAvailable: "security.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/security.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "template.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/template.openshift.io/v1?timeout=32s: context deadline exceeded (Client.Timeout exceeded while awaiting headers)) APIServicesAvailable: "user.openshift.io.v1" is not ready: 0 (Get https://10.205.0.1:443/apis/user.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers))' reason: APIServices_Error status: 'False' type: Available
-
Error "
Client.Timeout exceeded while awaiting headers
" in several pods:2021-01-29T03:28:34.330074737Z I0129 03:28:34.330006 1 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"c9ab82de-7fbe-11ea-ac8b-005056b2e4b3", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "oauth.openshift.io.v1" failed with HTTP status code 0 (Get https://10.205.0.1:443/apis/oauth.openshift.io/v1?timeout=32s: net/http: request canceled (Client.Timeout exceeded while awaiting headers))
Environment
- Red Hat OpenShift Container Platform (RHOCP, OCP)
- 4.5.16+
- 4.6
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.