hung tasks when accessing NetApp storage via multipath
Issue
- 'multipathd' process blocks frequently while accessing NetApp storage.
Feb 12 16:25:02 testsystem kernel: INFO: task multipathd:28497 blocked for more than 120 seconds.
Feb 12 16:25:02 testsystem kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 12 16:25:02 testsystem kernel: multipathd D 000000000000000a 0 28497 1 0x00000080
Feb 12 16:25:02 testsystem kernel: ffff880c6f469968 0000000000000086 0000000000000000 ffff880670412800
Feb 12 16:25:02 testsystem kernel: ffffffff81363180 ffff880c6df0ef00 ffff880c6f469938 ffffffffa0070e1e
Feb 12 16:25:02 testsystem kernel: ffff880c6d0abaf8 ffff880c6f469fd8 000000000000fb88 ffff880c6d0abaf8
Feb 12 16:25:02 testsystem kernel: Call Trace:
Feb 12 16:25:02 testsystem kernel: [<ffffffff81363180>] ? scsi_done+0x0/0x60
Feb 12 16:25:02 testsystem kernel: [<ffffffffa0070e1e>] ? qla2xxx_queuecommand+0x35e/0x370 [qla2xxx]
Feb 12 16:25:02 testsystem kernel: [<ffffffff814fe6a5>] schedule_timeout+0x215/0x2e0
Feb 12 16:25:02 testsystem kernel: [<ffffffff8107e0bd>] ? del_timer+0x7d/0xe0
Feb 12 16:25:02 testsystem kernel: [<ffffffff814fe323>] wait_for_common+0x123/0x180
Feb 12 16:25:02 testsystem kernel: [<ffffffff81060250>] ? default_wake_function+0x0/0x20
Feb 12 16:25:02 testsystem kernel: [<ffffffff814fe43d>] wait_for_completion+0x1d/0x20
Feb 12 16:25:02 testsystem kernel: [<ffffffff8125c8ac>] blk_execute_rq+0x8c/0xf0
Feb 12 16:25:02 testsystem kernel: [<ffffffff81054ce5>] ? select_idle_sibling+0x95/0x150
Feb 12 16:25:02 testsystem kernel: [<ffffffff81212ce9>] ? security_capable+0x29/0x30
Feb 12 16:25:02 testsystem kernel: [<ffffffff810787ea>] ? capable+0x2a/0x60
Feb 12 16:25:02 testsystem kernel: [<ffffffff81261092>] sg_io+0x212/0x3d0
Feb 12 16:25:02 testsystem kernel: [<ffffffff81261650>] scsi_cmd_ioctl+0x400/0x470
Feb 12 16:25:02 testsystem kernel: [<ffffffff81261711>] scsi_cmd_blk_ioctl+0x51/0x70
Feb 12 16:25:02 testsystem kernel: [<ffffffffa00e516f>] sd_ioctl+0xaf/0x110 [sd_mod]
Feb 12 16:25:02 testsystem kernel: [<ffffffff8125e357>] __blkdev_driver_ioctl+0x67/0x80
Feb 12 16:25:02 testsystem kernel: [<ffffffff8125e7dd>] blkdev_ioctl+0x1ed/0x6e0
Feb 12 16:25:02 testsystem kernel: [<ffffffff8126a197>] ? cic_free_func+0x87/0xa0
Feb 12 16:25:02 testsystem kernel: [<ffffffff811b381c>] block_ioctl+0x3c/0x40
Feb 12 16:25:02 testsystem kernel: [<ffffffff8118dec2>] vfs_ioctl+0x22/0xa0
Feb 12 16:25:02 testsystem kernel: [<ffffffff8118e064>] do_vfs_ioctl+0x84/0x580
Feb 12 16:25:02 testsystem kernel: [<ffffffff81070927>] ? do_exit+0x5b7/0x870
Feb 12 16:25:02 testsystem kernel: [<ffffffff810097cc>] ? __switch_to+0x1ac/0x320
Feb 12 16:25:02 testsystem kernel: [<ffffffff8118e5e1>] sys_ioctl+0x81/0xa0
Feb 12 16:25:02 testsystem kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Feb 12 16:26:31 testsystem kernel: qla2xxx [0000:07:00.0]-801c:2: Abort command issued nexus=2:0:0 -- 1 2002.
Feb 12 16:26:32 testsystem kernel: qla2xxx [0000:07:00.0]-801c:2: Abort command issued nexus=2:1:0 -- 1 2002.
Environment
- Red Hat Enterprise Linux 6.3
- NetApp storage
- device-mapper-multipath
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.