Openshift Upgrade Playbook fails when restarting node
Issue
- When starting the upgrade playbook, the play restart node fails with an error message.
- At this time, the
atomic-openshift-nodeservice on the first master has been tried to restart but fails to come up. - The logs shows:
Oct 18 11:41:46 node.example.com atomic-openshift-node[105325]: E1018 11:41:46.091066 105325 raw.go:140] Failed to watch directory "/sys/fs/cgroup/memory/system.slice/var-lib-docker-devicemapper-mnt-876aecbd6f58a637905da37ab34076ca55bd0950dbe0b5f91323e5d8937955c8.mount": inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-docker-devicemapper-mnt-876aecbd6f58a637905da37ab34076ca55bd0950dbe0b5f91323e5d8937955c8.mount: no space left on device
Oct 18 11:41:46 node.example.com atomic-openshift-node[105325]: E1018 11:41:46.091166 105325 raw.go:140] Failed to watch directory "/sys/fs/cgroup/memory/system.slice": inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-docker-devicemapper-mnt-876aecbd6f58a637905da37ab34076ca55bd0950dbe0b5f91323e5d8937955c8.mount: no space left on device
Oct 18 11:41:46 node.example.com atomic-openshift-node[105325]: F1018 11:41:46.091211 105325 kubelet.go:1352] Failed to start cAdvisor inotify_add_watch /sys/fs/cgroup/memory/system.slice/var-lib-docker-devicemapper-mnt-876aecbd6f58a637905da37ab34076ca55bd0950dbe0b5f91323e5d8937955c8.mount: no space left on device
Oct 18 11:41:46 node.example.com systemd[1]: atomic-openshift-node.service: main process exited, code=exited, status=255/n/a
Oct 18 11:41:46 node.example.com systemd[1]: Unit atomic-openshift-node.service entered failed state.
Oct 18 11:41:46 node.example.com systemd[1]: atomic-openshift-node.service failed.
Environment
- OpenShift Container Platform 3.11
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
