Collector pods have no requests and limits in OCP 4

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4
  • Red Hat Openshift Logging (RHOL)
    • 5

Issue

  • Collector pods consume excessive memory causing scheduling failures of critical components.
  • Collector pods have no limits and requests defined, even when they are not defined.

Resolution

Edit ClusterLogging custom resource (CR) instance in openshift-logging project deleting resources: {} in order to obtain default requests and limits values. If RHOL Operator is configured as Managed state, collector pods should be restarted automatically applying the new configuration :

$ oc patch ClusterLogging instance --type json -p '[{ "op": "remove", "path": "/spec/collection/logs/fluentd/resources" }]'

Check default resources have been configured in fluentd pods:

$ oc get pod <collector_pod> -o jsonpath='{.spec.containers[?(@.name=="fluentd")].resources}' 
{"limits":{"memory":"736Mi"},"requests":{"cpu":"100m","memory":"736Mi"}}

Root Cause

When the parameter /spec/collection/logs/fluentd/resources is defined like resources: {} in the ClusterLogging custom resource (CR) instance, then, the RHOL Operator interprets that the user is defining the requests and limits values, consequently, it doesn´t set the default.

Diagnostic Steps

Check if resources parameter is defined in ClusterLogging custom resource (CR) instance without content (limits and requests) like this:

$ oc get clusterlogging instance -o jsonpath='{.spec.collection.logs.fluentd.resources}'
{}

Check if resources parameter is defined in collector pod configuration without content (limits and requests) like this:

$ oc get pod <collector_pod> -o jsonpath='{.spec.containers[?(@.name=="fluentd")].resources}' 
{}

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments