OpenShift Node not starting due to "crio-wipe" error

Solution Verified - Updated -

Issue

  • When starting an OpenShift Container Platform Node, CRI-O fails to start with the error message Failed to shutdown storage before wiping: A layer is mounted: layer is in use by a container:

    Jan 12 15:17:59 example-001.example.com crio[1909]: time="2022-01-12T14:17:59Z" level=info msg="Starting CRI-O, version: 1.20.5-7.rhaos4.7.gite80c8db.el8, git: ()"
    Jan 12 15:17:59 example-001.example.com systemd[1]: Started libcontainer container 01edac1cfd0bb2a9b250a13e976f7373b7cbf1da5b5ab5d0415d8e81ee9cdf8b.
    Jan 12 15:17:59 example-001.example.com crio[1909]: time="2022-01-12 14:17:59.341782004Z" level=info msg="Checking whether cri-o should wipe containers: version file /var/run/crio/version not found: open /var/run/crio/version: no such file or directory"
    Jan 12 15:17:59 example-001.example.com crio[1909]: time="2022-01-12 14:17:59.345126179Z" level=info msg="File /var/lib/crio/clean.shutdown not found. Wiping storage directory /var/lib/containers/storage because of suspected dirty shutdown"
    Jan 12 15:17:59 example-001.example.com crio[1909]: time="2022-01-12 14:17:59.642582257Z" level=info msg="Failed to wipe storage cleanly: unlinkat /var/lib/containers/storage/overlay-containers/53a23eeb21c2e489e48b6e4b89c43b084af00758624de49bfc82ecfd31320210/userdata/shm: device or resource busy
    Jan 12 15:17:59 example-001.example.com crio[1909]: time="2022-01-25 14:17:59.643339908Z" level=fatal msg="Failed to shutdown storage before wiping: A layer is mounted: layer is in use by a container"
    Jan 12 15:17:59 example-001.example.com systemd[1]: crio-wipe.service: Main process exited, code=exited, status=1/FAILURE
    Jan 12 15:17:59 example-001.example.com systemd[1]: crio-wipe.service: Failed with result 'exit-code'.
    Jan 12 15:17:59 example-001.example.com systemd[1]: Failed to start CRI-O Auto Update Script.
    Jan 12 15:17:59 example-001.example.com systemd[1]: Dependency failed for Open Container Initiative Daemon.
    Jan 12 15:17:59 example-001.example.com systemd[1]: crio.service: Job crio.service/start failed with result 'dependency'.
    
  • When crio is starting, it fails with a device or resource busy error message.

Environment

  • Red Hat OpenShift Container Platform (RHOCP) 4.7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content