Cluster Operator unavailable with 'APIServicesAvailable: "apps.openshift.io.v1" is not ready'
Issue
-
The OpenShift Container Platform Cluster Operator
openshift-apiserver
is not available and is showing the followingAPIServices_Error
errors:APIServicesAvailable: "apps.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "authorization.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "build.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "image.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "project.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "quota.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "route.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "security.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request APIServicesAvailable: "template.openshift.io.v1" is not ready: an attempt failed with statusCode = 503, err = the server is currently unable to handle the request
-
The OpenShift API Server (
apiserver
) Pod in theopenshift-apiserver
namespace is showing the followingconnection refused
error messages:2023-06-19T07:59:22.838495962Z W0619 07:59:22.838437 1 logging.go:59] [core] [Channel #159 SubChannel #160] grpc: addrConn.createTransport failed to connect to { 2023-06-19T07:59:22.838495962Z "Addr": "10.140.20.23:2379", 2023-06-19T07:59:22.838495962Z "ServerName": "10.140.20.23", 2023-06-19T07:59:22.838495962Z "Attributes": null, 2023-06-19T07:59:22.838495962Z "BalancerAttributes": null, 2023-06-19T07:59:22.838495962Z "Type": 0, 2023-06-19T07:59:22.838495962Z "Metadata": null 2023-06-19T07:59:22.838495962Z }. Err: connection error: desc = "transport: Error while dialing dial tcp 10.140.20.23:2379: connect: connection refused"
-
The
etcd
Pods show error messages similar to the following:2023-06-19T09:34:37.366653439Z {"level":"warn","ts":"2023-06-19T09:34:37.366Z","caller":"rafthttp/http.go:394","msg":"rejected stream from remote peer because it was removed","local-member-id":"ed898fb22814c213","remote-peer-id-stream-handler":"fd898fb22814c212","remote-peer-id-from":"32b626d55668f0af"} 2023-06-19T09:34:37.367468970Z {"level":"warn","ts":"2023-06-19T09:34:37.367Z","caller":"rafthttp/http.go:394","msg":"rejected stream from remote peer because it was removed","local-member-id":"ed898fb22814c213","remote-peer-id-stream-handler":"fd898fb22814c212","remote-peer-id-from":"32b626d55668f0af"}
Environment
- Red Hat OpenShift Container Platform (RHOCP) 4.12
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.