What does the error "failed to send out heartbeat on time" in etcd logs mean?
Issue
- The following logs are appearing in etcd pod:
W | etcdserver: failed to send out heartbeat on time (exceeded the 100ms timeout for 105.137408ms)
W | etcdserver: server is likely overloaded
W | etcdserver: read-only range request "key:\"/kubernetes.io/configmaps/openshift-kube-scheduler/scheduler-kubeconfig\" " with result "range_response_count:1 size:867" took too long (403.657982ms) to execute
W | etcdserver: read-only range request "key:\"/kubernetes.io/leases/kube-node-lease/cluster-qjrwn-worker-brazilsouth-fvq4d\" " with result "range_response_count:1 size:419" took too long (394.615802ms) to execute
W | etcdserver: read-only range request "key:\"/kubernetes.io/leases/kube-node-lease/cluster-qjrwn-worker-brazilsouth-76bhv\" " with result "range_response_count:1 size:420" took too long (391.729576ms) to execute
- These messages can either be intermittent and not frequent, or constant.
Environment
- OpenShift Container Platform
- 4.x
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.