Why is cpu usage of dmeventd high in connection with full Docker thin pool?

Solution Verified - Updated -

Issue

If the current allocation of the thin pool gets close to the total amount of space available in the VG, each attempt to extend the pool will fail and will be logged in /var/log/messages similar to the following example.

  Thin docker--storage-docker--pool is now 99% full.
  Insufficient free space: 510 extents needed, but only 6 available
  Failed to extend thin docker--storage-docker--pool.

The thin pool allocation is monitored by dmeventd which retries to extend the pool once every 10 seconds by emulating an lvextend command (dmeventd does not fork/exec a separate process). For example, it emulates

  lvextend --use-policies docker-storage/docker-pool

During each attempted/failed lvextend operation, an additional file gets created in the /etc/lvm/archive directory. Hence, if a system administator does not take corrective action by providing more space to the thin pool, the number of files in /etc/lvm/archive can grow substantially over a long period of time (one additional file every 10 seconds results in 8640 files every day). And the huge number of files in the /etc/lvm/archive directory would cause the high cpu usage of dmeventd.

Environment

  • Docker with lvm thin pool devicemapper
  • Prior to lvm2-2.02.166-1.el7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content