RHEL 6.5 - "Kernel: multipath, not tainted" alarms
Hi, this is my first post on RHEL forums! I feel the need to post, as since the most recent kernel patch I am seeing the following errors:
Feb 10 05:33:18 know-iprd-testaql1 kernel: INFO: task multipathd:4620 blocked for more than 120 seconds. Feb 10 05:33:18 know-iprd-testaql1 kernel: Not tainted 2.6.32-431.3.1.el6.x86_64 #1 Feb 10 05:33:18 know-iprd-testaql1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 10 05:33:18 know-iprd-testaql1 kernel: multipathd D 0000000000000005 0 4620 1 0x00000000 Feb 10 05:33:18 know-iprd-testaql1 kernel: ffff88105ac4f968 0000000000000082 0000000000000000 ffff880b48cf8880 Feb 10 05:33:18 know-iprd-testaql1 kernel: ffff881061c305e0 ffff881063c21000 ffffffff8137ffe0 ffff880b48ebee40 Feb 10 05:33:18 know-iprd-testaql1 kernel: ffff88105a828638 ffff88105ac4ffd8 000000000000fbc8 ffff88105a828638 Feb 10 05:33:18 know-iprd-testaql1 kernel: Call Trace: Feb 10 05:33:18 know-iprd-testaql1 kernel: [] ? scsi_done+0x0/0x60 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] schedule_timeout+0x215/0x2e0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] ? del_timer+0x7d/0xe0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] wait_for_common+0x123/0x180 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] ? default_wake_function+0x0/0x20 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] wait_for_completion+0x1d/0x20 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] blk_execute_rq+0x8c/0xf0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] ? blk_rq_map_user+0x1a0/0x280 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] sg_io+0x215/0x3d0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] scsi_cmd_ioctl+0x400/0x470 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] scsi_cmd_blk_ioctl+0x51/0x70 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] sd_ioctl+0xaf/0x110 [sd_mod] Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] __blkdev_driver_ioctl+0x67/0x80 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] blkdev_ioctl+0x1ed/0x6e0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] block_ioctl+0x3c/0x40 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] vfs_ioctl+0x22/0xa0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] ? _atomic_dec_and_lock+0x55/0x80 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] do_vfs_ioctl+0x84/0x580 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] ? __fput+0x1a1/0x210 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] sys_ioctl+0x81/0xa0 Feb 10 05:33:18 know-iprd-testaql1 kernel: [ ] system_call_fastpath+0x16/0x1b Feb 10 05:35:18 know-iprd-testaql1 kernel: INFO: task multipathd:4620 blocked for more than 120 seconds. Feb 10 05:35:18 know-iprd-testaql1 kernel: Not tainted 2.6.32-431.3.1.el6.x86_64 #1 Feb 10 05:35:18 know-iprd-testaql1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 10 05:35:18 know-iprd-testaql1 kernel: multipathd D 0000000000000005 0 4620 1 0x00000000 Feb 10 05:35:18 know-iprd-testaql1 kernel: ffff88105ac4f968 0000000000000082 0000000000000000 ffff880b48cf8880 Feb 10 05:35:18 know-iprd-testaql1 kernel: ffff881061c305e0 ffff881063c21000 ffffffff8137ffe0 ffff880b48ebee40 Feb 10 05:35:18 know-iprd-testaql1 kernel: ffff88105a828638 ffff88105ac4ffd8 000000000000fbc8 ffff88105a828638 Feb 10 05:35:18 know-iprd-testaql1 kernel: Call Trace: Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? scsi_done+0x0/0x60 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] schedule_timeout+0x215/0x2e0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? del_timer+0x7d/0xe0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] wait_for_common+0x123/0x180 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? default_wake_function+0x0/0x20 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] wait_for_completion+0x1d/0x20 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] blk_execute_rq+0x8c/0xf0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? blk_rq_map_user+0x1a0/0x280 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] sg_io+0x215/0x3d0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] scsi_cmd_ioctl+0x400/0x470 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] scsi_cmd_blk_ioctl+0x51/0x70 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] sd_ioctl+0xaf/0x110 [sd_mod] Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] __blkdev_driver_ioctl+0x67/0x80 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] blkdev_ioctl+0x1ed/0x6e0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] block_ioctl+0x3c/0x40 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] vfs_ioctl+0x22/0xa0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? _atomic_dec_and_lock+0x55/0x80 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] do_vfs_ioctl+0x84/0x580 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] ? __fput+0x1a1/0x210 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] sys_ioctl+0x81/0xa0 Feb 10 05:35:18 know-iprd-testaql1 kernel: [ ] system_call_fastpath+0x16/0x1b Feb 10 05:36:17 know-iprd-testaql1 multipathd: sdq: couln't get asymmetric access state Feb 10 05:36:17 know-iprd-testaql1 kernel: qla2xxx [0000:06:00.0]-801c:0: Abort command issued nexus=0:1:4 -- 1 2002. Feb 10 05:36:17 know-iprd-testaql1 multipathd: data04: load table [0 419430400 multipath 1 queue_if_no_path 0 3 2 round-robin 0 1 1 65:0 1 round-robin 0 4 1 65:192 1 66:128 1 67:64 1 68:0 1 round-robin 0 3 1 8:64 1 68:192 1 69:128 1] Feb 10 05:36:37 know-iprd-testaql1 multipathd: data04: load table [0 419430400 multipath 1 queue_if_no_path 0 2 1 round-robin 0 4 1 65:192 1 66:128 1 67:64 1 68:0 1 round-robin 0 4 1 65:0 1 8:64 1 68:192 1 69:128 1]
I don't know if this was an issue prior to the patch, or not, sorry. What I do know is, it worries me....should it?
What is actually broken here, if anything? Is there a fix?
Many thanks,
Mark Hands
Responses