etcd backend performance requirements

Solution Verified - Updated -

Issue

When deploying OpenShift cluster on top of virtualized user-provided infrastructure (OpenStack/VMWare), etcd performance can be impacted by poor storage and network performance, causing multiple errors happened:

$ oc logs --follow=true etcd-ocp4-9wwcf-master-0 -c etcd -n openshift-etcd
...
etcdserver: failed to send out heartbeat on time (exceeded the 100ms timeout for xxx ms)
etcdserver: server is likely overloaded
etcdserver: read-only range request "key:\"xxxx" count_only:true " with result "xxxx" took too long (xxx s) to execute
etcdserver: read-only range request "key:\"xxxx" count_only:true " with result "xxxx" took too long (xxxx ms) to execute
etcdserver: read-only range request "xxxx" with result "xxxx" took too long (xxx ms) to execute
wal: sync duration of xxxx s, expected less than 1s

Environment

  • OpenShift Container Platform
    • 3.x
    • 4.x
    • User-provided virtualized infrastructure

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In