System deadlocks while taking LVM snapshot

Solution Unverified - Updated -

Issue

  • System deadlocks while taking LVM snapshot
Apr 21 13:47:07 host kernel: INFO: task kworker/1:1:48 blocked for more than 120 seconds.
Apr 21 13:47:07 host kernel:      Tainted: P           OE    --------- -  - 4.18.0-147.5.1.el8_1.x86_64 #1
Apr 21 13:47:07 host kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 21 13:47:07 host kernel: kworker/1:1     D    0    48      2 0x80000000
Apr 21 13:47:07 host kernel: Workqueue: kcopyd do_work [dm_mod]
Apr 21 13:47:07 host kernel: Call Trace:
Apr 21 13:47:07 host kernel: ? __schedule+0x253/0x830
Apr 21 13:47:07 host kernel: ? dd_finish_request+0x40/0x40
Apr 21 13:47:07 host kernel: schedule+0x28/0x70
Apr 21 13:47:07 host kernel: rwsem_down_read_failed+0x10d/0x1b0
Apr 21 13:47:07 host kernel: do_origin.isra.28+0x19/0x90 [dm_snapshot]
Apr 21 13:47:07 host kernel: pending_complete+0x22b/0x390 [dm_snapshot]
Apr 21 13:47:07 host kernel: persistent_commit_exception+0xce/0x140 [dm_snapshot]
Apr 21 13:47:07 host kernel: copy_callback+0x9c/0x170 [dm_snapshot]
Apr 21 13:47:07 host kernel: ? start_copy+0xd0/0xd0 [dm_snapshot]
Apr 21 13:47:07 host kernel: run_complete_job+0x5f/0xa0 [dm_mod]
Apr 21 13:47:07 host kernel: process_jobs+0x89/0x210 [dm_mod]
Apr 21 13:47:07 host kernel: ? drop_pages+0x40/0x40 [dm_mod]
Apr 21 13:47:07 host kernel: do_work+0x93/0xf0 [dm_mod]
Apr 21 13:47:07 host kernel: process_one_work+0x1a7/0x3b0
Apr 21 13:47:07 host kernel: worker_thread+0x1fa/0x390
Apr 21 13:47:07 host kernel: ? create_worker+0x1a0/0x1a0
Apr 21 13:47:07 host kernel: kthread+0x112/0x130
Apr 21 13:47:07 host kernel: ? kthread_flush_work_fn+0x10/0x10
Apr 21 13:47:07 host kernel: ret_from_fork+0x35/0x40

Apr 21 13:47:07 host kernel: INFO: task kworker/u12:27:659 blocked for more than 120 seconds.
Apr 21 13:47:07 host kernel: Call Trace:
Apr 21 13:47:07 host kernel: ? __schedule+0x253/0x830
Apr 21 13:47:07 host kernel: schedule+0x28/0x70
Apr 21 13:47:07 host kernel: schedule_timeout+0x26d/0x390
Apr 21 13:47:07 host kernel: ? mempool_alloc+0x67/0x190
Apr 21 13:47:07 host kernel: ? kmem_cache_alloc+0x158/0x1c0
Apr 21 13:47:07 host kernel: ? finish_wait+0x80/0x80
Apr 21 13:47:07 host kernel: ? mempool_alloc+0x67/0x190
Apr 21 13:47:07 host kernel: __down+0x9b/0xf0
Apr 21 13:47:07 host kernel: down+0x3b/0x50
Apr 21 13:47:07 host kernel: start_copy+0x8a/0xd0 [dm_snapshot]
Apr 21 13:47:07 host kernel: __origin_write+0x1b8/0x350 [dm_snapshot]
Apr 21 13:47:07 host kernel: do_origin.isra.28+0x7e/0x90 [dm_snapshot]
Apr 21 13:47:07 host kernel: __map_bio+0x42/0x190 [dm_mod]
Apr 21 13:47:07 host kernel: __split_and_process_non_flush+0x14b/0x1b0 [dm_mod]
Apr 21 13:47:07 host kernel: __split_and_process_bio+0x86/0x230 [dm_mod]
Apr 21 13:47:07 host kernel: ? blk_queue_split+0x3bf/0x640
Apr 21 13:47:07 host kernel: ? blk_queue_split+0xdd/0x640
Apr 21 13:47:07 host kernel: dm_process_bio+0x15a/0x220 [dm_mod]
Apr 21 13:47:07 host kernel: ? generic_make_request_checks+0x1fd/0x660
Apr 21 13:47:07 host kernel: dm_make_request+0x3b/0x90 [dm_mod]
Apr 21 13:47:07 host kernel: generic_make_request+0xcf/0x310
Apr 21 13:47:07 host kernel: submit_bio+0x45/0x140
Apr 21 13:47:07 host kernel: ? xfs_setfilesize_trans_alloc.isra.15+0x3d/0x90 [xfs]
Apr 21 13:47:07 host kernel: xfs_submit_ioend+0x97/0x1d0 [xfs]
Apr 21 13:47:07 host kernel: xfs_vm_writepages+0x78/0xa0 [xfs]
Apr 21 13:47:07 host kernel: do_writepages+0x41/0xd0
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x41/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x35/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x41/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x35/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x41/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x35/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x41/0x70
Apr 21 13:47:07 host kernel: ? __switch_to_asm+0x41/0x70
Apr 21 13:47:07 host kernel: __writeback_single_inode+0x3d/0x360
Apr 21 13:47:07 host kernel: writeback_sb_inodes+0x1e3/0x45

Environment

  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise Linux 8

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content