Ceph kernel RBD: System hanging with tasks stuck in XFS

Solution Unverified - Updated -

Issue

  • Ceph kernel RBD: System hanging with tasks stuck in XFS.
  • Call trace from /var/log/messages:

    kernel: INFO: task java:31844 blocked for more than 120 seconds.
    kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    kernel: java            D 0000000000000000     0 31844      1 0x00000080
    kernel: ffff883913f0f6e0 0000000000000082 ffff884458bb6780 ffff883913f0ffd8
    kernel: ffff883913f0ffd8 ffff883913f0ffd8 ffff884458bb6780 ffff88227673ebb0
    kernel: 7fffffffffffffff ffff884458bb6780 0000000000000002 0000000000000000
    kernel: Call Trace:
    kernel: [] schedule+0x29/0x70
    kernel: [] schedule_timeout+0x209/0x2d0
    kernel: [] ? xfs_buf_rele+0x40/0x100 [xfs]
    kernel: [] ? xfs_buf_readahead_map+0x4f/0x70 [xfs]
    kernel: [] __down_common+0xd2/0x14a
    kernel: [] ? _xfs_buf_find+0x16d/0x2c0 [xfs]
    kernel: [] __down+0x1d/0x1f
    kernel: [] down+0x41/0x50
    kernel: [] xfs_buf_lock+0x3c/0xd0 [xfs]
    kernel: [] _xfs_buf_find+0x16d/0x2c0 [xfs]
    kernel: [] xfs_buf_get_map+0x2a/0x180 [xfs]
    kernel: [] xfs_buf_read_map+0x2c/0x140 [xfs]
    kernel: [] xfs_trans_read_buf_map+0x211/0x400 [xfs]
    kernel: [] xfs_read_agi+0x9d/0x110 [xfs]
    kernel: [] xfs_ialloc_read_agi+0x34/0xd0 [xfs]
    kernel: [] xfs_dialloc+0xe8/0x280 [xfs]
    kernel: [] xfs_ialloc+0x71/0x540 [xfs]
    kernel: [] ? kmem_zone_alloc+0x77/0x100 [xfs]
    

Environment

  • Red Hat Enterprise Linux 7.1
  • Red Hat Enterprise Linux 7.2
  • Ceph Kernel RBD Driver

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In