Pods cannot be started in some nodes after Symantec Endpoint Protection has been installed

Solution Verified - Updated -

Issue

Some nodes are not able to host pods anymore. They switch between Ready and Not ready status. For example:

 Oct 21 19:04:56 node01.redhat.com atomic-openshift-node[48147]: I1021 19:04:56.836057   48147 kubelet.go:1758] skipping pod synchronization - [PLEG is not heal
 thy: pleg was last seen active 7m18.734318929s ago; threshold is 3m0s]
 Oct 21 19:04:57 node01.redhat.com sshd[69124]: pam_unix(sshd:session): session opened for user slwsvc by (uid=0)
 Oct 21 19:05:01 node01.redhat.com atomic-openshift-node[48147]: I1021 19:05:01.836454   48147 kubelet.go:1758] skipping pod synchronization - [PLEG is not healthy: pleg was last seen active 7m23.734736911s ago; threshold is 3m0s]
 Oct 21 19:05:04 node01.redhat.com kernel: device-mapper: thin: Deletion of thin device 74176 failed.
 Oct 21 19:05:04 node01.redhat.com dockerd-current[30020]: time="2019-10-21T19:05:04.207931262+07:00" level=error msg="Handler for POST /v1.26/containers/9c7b649f434a

While at the controllers, we can see it switching between Ready and notReady:

 I1021 19:05:46.069794       1 node_lifecycle_controller.go:624] Node is NotReady. Adding Pods on Node node01.redhat.com to eviction queue: 2019-10-21 19:05:46.069753008 +0700 WIB m=+1527792.552778618 is later than 2019-10-21 19:00:45.926291774 +0700 WIB m=+1527492.409317407 + 5m0s
 I1021 19:05:46.356961       1 controller_utils.go:212] Recording Deleting all Pods from Node node01.redhat.com. event message for node node01.redhat.com

 [...]

 I1021 19:08:01.131389       1 node_lifecycle_controller.go:1108] Cancelling pod Eviction on Node: node01.redhat.com
 I1021 19:08:01.131484       1 node_lifecycle_controller.go:672] Node node01.redhat.com is ready again, cancelled pod eviction

Environment

OCP 3.11
Docker 1.13.1

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content