Moving Infra Components to Master / Control Plane Nodes for OCP 4.x

Issue

Aa large number of components critical to the cluster operability are isolated on the masters, there are still some that by default run on the worker nodes.

These services include:

• Routers
• Registries
• Monitoring
• Logging

In addition to these, each organization should analyze which cluster additions are viewed as critical for their needs.

If the master nodes are large enough to accommodate all critical resources (ex. the nodes are big bare metal servers) moving infra workloads on the control plane is possible. But, since OpenShift is not configured to allow this by default, additional steps must be taken.

When moving infrastructure components to the master nodes, the following caveats should be considered:
* I/O hungry infrastucture components should be avoided on master nodes. This is because etcd running on the master nodes is very sensitive to disk latency
* having infra workloads on the master nodes can increase the reboot time of these nodes. For example, the ingress controller pods can cause slow node reboots due to high terminationGracePeriodSeconds.
* if possible set ressource limits on the infra workload, for instance for monitoring you can follow How to set up the resource limits and requests for Prometheus in OpenShift ?