The kernel is getting hung upon inode writeback. A possible deadlock caused by a 3rd party module named "dtfilter"

Solution Unverified - Updated -

Issue

  • The kernel is getting hung upon inode writeback. A possible deadlock caused by a 3rd party module named "dtfilter"
[3481104.495210] INFO: task kworker/u16:2:27126 blocked for more than 120 seconds.
[3481104.495540] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[3481104.495853] kworker/u16:2   D ffff8e44ffd9ac80     0 27126      2 0x00000080
[3481104.495864] Workqueue: writeback bdi_writeback_workfn (flush-253:5)
[3481104.495867] Call Trace:
[3481104.495876]  [<ffffffffacc264d5>] ? __kmalloc+0x55/0x230
[3481104.495885]  [<ffffffffc08e402b>] ? QueueItemMemAlloc.isra.0+0x2b/0xa0 [dtrep]
[3481104.495892]  [<ffffffffad17eb20>] ? out_of_line_wait_on_atomic_t+0x100/0x100
[3481104.495895]  [<ffffffffad180a29>] schedule+0x29/0x70
[3481104.495898]  [<ffffffffad17eb31>] bit_wait+0x11/0x50
[3481104.495901]  [<ffffffffad17e6a7>] __wait_on_bit+0x67/0x90
[3481104.495919]  [<ffffffffacc797a2>] __inode_wait_for_writeback+0xb2/0xf0
[3481104.495924]  [<ffffffffacac73a0>] ? wake_bit_function+0x40/0x40
[3481104.495927]  [<ffffffffacc7b556>] inode_wait_for_writeback+0x26/0x40
[3481104.495931]  [<ffffffffacc685af>] evict+0x9f/0x180
[3481104.495933]  [<ffffffffacc689ec>] iput+0xfc/0x190
[3481104.495936]  [<ffffffffacc636f8>] __dentry_kill+0x128/0x190
[3481104.495938]  [<ffffffffacc63816>] dput+0xb6/0x1a0
[3481104.495953]  [<ffffffffc07256e2>] superWriteInodeHook+0x1b2/0x230 [dtfilter]
[3481104.495956]  [<ffffffffacc79a2d>] __writeback_single_inode+0x1ad/0x260
[3481104.495958]  [<ffffffffacc7a454>] writeback_sb_inodes+0x1c4/0x430
[3481104.495961]  [<ffffffffacc7a75f>] __writeback_inodes_wb+0x9f/0xd0
[3481104.495963]  [<ffffffffacc7ac43>] wb_writeback+0x263/0x2f0
[3481104.495967]  [<ffffffffacbca360>] ? bdi_dirty_limit+0x40/0xe0
[3481104.495969]  [<ffffffffacc7b73c>] bdi_writeback_workfn+0x1cc/0x460
[3481104.495972]  [<ffffffffacabe21f>] process_one_work+0x17f/0x440
[3481104.495975]  [<ffffffffacabf336>] worker_thread+0x126/0x3c0
[3481104.495977]  [<ffffffffacabf210>] ? manage_workers.isra.26+0x2a0/0x2a0
[3481104.495980]  [<ffffffffacac61f1>] kthread+0xd1/0xe0
[3481104.495982]  [<ffffffffacac6120>] ? insert_kthread_work+0x40/0x40
[3481104.495991]  [<ffffffffad18dd37>] ret_from_fork_nospec_begin+0x21/0x21
[3481104.495994]  [<ffffffffacac6120>] ? insert_kthread_work+0x40/0x40

Environment

  • Red Hat Enterprise Linux 7.7 (kernel-3.10.0-1062.12.1.el7.x86_64)
    • The issue could happen with other rhel7 minor releases
  • dtfilter whcih is one of modules from the Double-Take product packaged as "Carbonite Availability"

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content