OpenShift rpm-ostree stuck in UpdateDeployment

Solution Unverified - Updated -

Issue

Node become stuck during MachineConfigDaemon (MCD) update due to an issue with rpm-ostree not completing successfully. Logs will show what looks like an infinte loop of starting and stopping of the rpm-ostree service.

Jul 14 13:09:16 worker05 root[2753771]: machine-config-daemon[2610060]: Starting update from rendered-worker-0c4a7b6581b0065d403c2c7e9589eab2 to rendered-worker-efe5644e2541c01d1fe192c364b40cf5: &{osUpdate:true kargs:false fips:false passwd:false files:true units:true kernelType:false extensions:false}
Jul 14 13:09:16 worker05 root[2753772]: machine-config-daemon[2610060]: Update prepared; beginning drain
Jul 14 13:09:16 worker05 root[2753791]: machine-config-daemon[2610060]: drain complete
Jul 14 13:09:30 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111674 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) added; new total=1
Jul 14 13:09:30 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111674 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) vanished; remaining=0
Jul 14 13:09:30 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111675 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) added; new total=1
Jul 14 13:09:30 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111675 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) vanished; remaining=0
Jul 14 13:10:30 worker05 root[2754611]: machine-config-daemon[2610060]: Starting update from rendered-worker-0c4a7b6581b0065d403c2c7e9589eab2 to rendered-worker-efe5644e2541c01d1fe192c364b40cf5: &{osUpdate:true kargs:false fips:false passwd:false files:true units:true kernelType:false extensions:false}
Jul 14 13:10:30 worker05 root[2754612]: machine-config-daemon[2610060]: Update prepared; beginning drain
Jul 14 13:10:31 worker05 root[2754619]: machine-config-daemon[2610060]: drain complete
Jul 14 13:11:10 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111676 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) added; new total=1
Jul 14 13:11:10 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111676 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) vanished; remaining=0
Jul 14 13:11:11 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111677 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) added; new total=1
Jul 14 13:11:11 worker05 rpm-ostree[2689365]: client(id:machine-config-operator dbus:1.111677 unit:crio-6ef8ee933b95235c515b6109cb6f7aba203e92e3e87c692703bfbb98c5225e2a.scope uid:0) vanished; remaining=0
Jul 14 13:12:11 worker05 root[2755785]: machine-config-daemon[2610060]: Starting update from rendered-worker-0c4a7b6581b0065d403c2c7e9589eab2 to rendered-worker-efe5644e2541c01d1fe192c364b40cf5: &{osUpdate:true kargs:false fips:false passwd:false files:true units:true kernelType:false extensions:false}
Jul 14 13:12:11 worker05 root[2755786]: machine-config-daemon[2610060]: Update prepared; beginning drain
Jul 14 13:12:12 worker05 root[2755795]: machine-config-daemon[2610060]: drain complete

Environment

OpenShift Container Platform

  • 4.x

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content