System kernel panic, appears "kernel BUG at /SFHA_build/fsbuild/6.0.1/6.0.500.100/2014-08-27_23.00.19/linux/vxfs/non-debug/rhel6/x86/kernel/smp/vx_strategy.c:1481!"

Solution Unverified - Updated -

Issue

  • System crashed inside vxfs vx_dev_strategy with kernel BUG statement similar to the following
<2>kernel BUG at /SFHA_build/fsbuild/6.0.1/6.0.500.200/2014-12-04_16.07.02/linux/vxfs/non-debug/rhel6/x86/kernel/smp_u6/vx_strategy.c:1481!
<4>invalid opcode: 0000 [#1] SMP 
<4>last sysfs file: /sys/devices/pci0000:60/0000:60:05.4/irq
<4>CPU 54 
<4>Modules linked in: dm_snapshot dm_bufio bridge mptctl mptbase vxodm(P)(U) vxgms(P)(U) amf(P)(U) vxglm(P)(U) vxfen(P)(U) gab(P)(U) llt(P)(U) autofs4 nfs lockd fscache auth_rpcgss nfs_acl sunrpc bonding 8021q garp stp llc ipv6 vxportal(P)(U) fdd(P)(U) vxfs(P)(U) exportfs iTCO_wdt iTCO_vendor_support microcode serio_raw hpilo hpwdt ixgbe ptp pps_core mdio lpc_ich mfd_core ioatdma dca sg power_meter acpi_ipmi ipmi_si ipmi_msghandler shpchp ext4 jbd2 mbcache dmpjbod(P)(U) dmpap(P)(U) dmpaa(P)(U) vxspec(P)(U) vxio(P)(U) vxdmp(P)(U) sd_mod hpsa lpfc scsi_transport_fc scsi_tgt crc_t10dif dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
<4>
<4>Pid: 19359, comm: multitool Tainted: P           ---------------    2.6.32-504.8.1.el6.x86_64 #1 HP ProLiant DL560 Gen8
<4>RIP: 0010:[<ffffffffa08c094e>]  [<ffffffffa08c094e>] vx_dev_strategy+0x68e/0x6f0 [vxfs]
<4>RSP: 0000:ffff88072dfe0798  EFLAGS: 00010297
<4>RAX: ffff88072dfe0000 RBX: 0000000000000600 RCX: ffff8801e6e52aa0
<4>RDX: ffff884040018e68 RSI: ffff88400f6ecae0 RDI: ffff886012ab7bc0
<4>RBP: ffff88072dfe0888 R08: 000000000008f664 R09: 000000000c70afc8
<4>R10: 000000000000001b R11: 0000000000003000 R12: 0000000000000000
<4>R13: 0000000000000000 R14: ffffea000c0c67a8 R15: 0000000000000000
<4>FS:  00007f9a0b94e720(0000) GS:ffff8840b0dc0000(0000) knlGS:0000000000000000
<4>CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>CR2: 00007f9a0b536000 CR3: 000000600ee32000 CR4: 00000000000407e0
<4>DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>Process multitool (pid: 19359, threadinfo ffff88072dfe0000, task ffff8801e6e52aa0)
<4>Stack:
<4> ffff88072dfe07a8 0000000000001609 ffff880371420000 0000000000000000
<4><d> 0000000000000001 0000000000000000 00000001ffffffff ffff8801e6e52aa0
<4><d> ffff8801e6e52aa0 ffff88200c013a00 ffff8801e6e52aa0 0000000000000000
<4>Call Trace:
<4> [<ffffffffa08c0f05>] vx_snap_strategy+0x295/0x2d0 [vxfs]
<4> [<ffffffffa08356fd>] vx_logbuf_write+0x17d/0x1b0 [vxfs]
<4> [<ffffffffa083590f>] vx_logbuf_io+0x1df/0x340 [vxfs]
<4> [<ffffffffa08366ae>] vx_logbuf_flush+0x22e/0x2d0 [vxfs]
<4> [<ffffffffa0836832>] vx_logflush+0xe2/0x140 [vxfs]
  • Also found below similar kind of trace
BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
IP: [<ffffffffa050cde5>] vx_dev_strategy+0x495/0x7c0 [vxfs]
PGD 15f146067 PUD faa586067 PMD 0 
Thread overran stack, or stack corrupted
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/pci0000:80/0000:80:01.0/0000:81:00.1/temp1_input
CPU 11 
[..]
Pid: 42600, comm: java Tainted: P           ---------------    2.6.32-504.el6.x86_64 #1 HP ProLiant DL380 Gen9/ProLiant DL380 Gen9
RIP: 0010:[<ffffffffa050cde5>]  [<ffffffffa050cde5>] vx_dev_strategy+0x495/0x7c0 [vxfs]
[..]
Call Trace:
 [<ffffffffa050d67d>] vx_snap_strategy+0x2ed/0x330 [vxfs]
 [<ffffffffa04f85bc>] vx_io_startnowait+0x10c/0x3c0 [vxfs]
 [<ffffffffa0403b66>] ? vx_io_setup+0x56/0xe0 [vxfs]
 [<ffffffffa04fce4b>] vx_nalloc_getpage_lnx+0xa8b/0x14b0 [vxfs]
 [<ffffffff81146d5e>] ? __inc_zone_page_state+0x2e/0x30
 [<ffffffffa0c98e00>] ? vxg_range_lockwf_fastsub+0x60/0x140 [vxglm]
 [<ffffffffa0407a59>] vx_do_getpage+0x1c19/0x2500 [vxfs]
 [<ffffffffa0c85c00>] ? vxg_lock_fastsub+0x0/0xe0 [vxglm]
 [<ffffffffa030f4f8>] ? vx_sum_upd_efree1+0x88/0xa0 [vxfs]
 [<ffffffffa04ad400>] ? vx_alloc+0x8b0/0xd10 [vxfs]
 [<ffffffffa051108e>] ? vx_subtranalloc+0xde/0x270 [vxfs]
 [<ffffffffa04d7208>] vx_do_read_ahead+0x1a8/0x220 [vxfs]
 [<ffffffffa0453aca>] vx_read_ahead+0x3ca/0xed0 [vxfs]
 [<ffffffffa0409505>] ? vx_glm_range_lock+0x25/0x50 [vxfs]
 [<ffffffffa0403543>] ? vx_glmrange_rangelock+0x33/0x90 [vxfs]
 [<ffffffffa035d581>] ? vx_ipglock+0x31/0x40 [vxfs]
 [<ffffffffa04052d2>] ? vx_getpage_rangelock+0x142/0x150 [vxfs]
 [<ffffffffa04067a1>] vx_do_getpage+0x961/0x2500 [vxfs]
 [<ffffffffa04036f1>] ? vx_genglm_lock+0x31/0x40 [vxfs]
 [<ffffffffa0435400>] ? vx_iglock3+0x70/0x110 [vxfs]
 [<ffffffff8100988e>] ? __switch_to+0x26e/0x320
 [<ffffffffa0408741>] vx_getpage1+0x401/0x940 [vxfs]
 [<ffffffff8117d170>] ? mem_cgroup_get_reclaim_stat_from_page+0x20/0x70
 [<ffffffffa04b1651>] vx_fault+0x2c1/0x6c0 [vxfs]
 [<ffffffff811807d7>] ? mem_cgroup_update_file_mapped+0x17/0x90
 [<ffffffff8114eae4>] __do_fault+0x54/0x530
 [<ffffffff8114f0b7>] handle_pte_fault+0xf7/0xb00
 [<ffffffff81182028>] ? mem_cgroup_cache_charge+0xb8/0xc0
 [<ffffffffa0c98ef2>] ? vxg_range_lock_fastsub+0x12/0x40 [vxglm]
 [<ffffffffa0c98fcc>] ? vxg_range_unlock_fastsub+0x2c/0x40 [vxglm]
 [<ffffffffa0c97ded>] ? vxg_lock_ilock_omnibus+0x3d/0x2a0 [vxglm]
 [<ffffffff8114fcea>] handle_mm_fault+0x22a/0x300
 [<ffffffff8114feea>] __get_user_pages+0x12a/0x430
 [<ffffffffa0c88064>] ? vxg_api_lock+0x54/0xa0 [vxglm]
 [<ffffffff81150289>] get_user_pages+0x49/0x50
 [<ffffffffa04a5ab1>] vx_prefault_uio+0x1c1/0x2c0 [vxfs]
 [<ffffffffa0409000>] ? vx_glm_trylock+0x30/0x80 [vxfs]
 [<ffffffffa0539572>] vx_write_default+0x3e2/0x1610 [vxfs]
 [<ffffffffa05343a9>] ? vx_write_alloc+0x409/0x6e0 [vxfs]
 [<ffffffffa053ae41>] vx_write1+0x6a1/0x1940 [vxfs]
 [<ffffffffa0c88064>] ? vxg_api_lock+0x54/0xa0 [vxglm]
 [<ffffffffa0c85c00>] ? vxg_lock_fastsub+0x0/0xe0 [vxglm]
 [<ffffffffa0c90001>] ? vxg_recv_release+0x61/0x160 [vxglm]
 [<ffffffffa053c41f>] vx_write_common_slow+0x33f/0xfd0 [vxfs]
 [<ffffffffa053d887>] vx_write_common+0x5f7/0x9f0 [vxfs]
 [<ffffffff8152d375>] ? page_fault+0x25/0x30
 [<ffffffffa04a5248>] ? vx_prefault_uio_readable+0x98/0xf0 [vxfs]
 [<ffffffffa04b9c7a>] vx_write+0x20a/0x370 [vxfs]
 [<ffffffff81156225>] ? do_mmap_pgoff+0x335/0x380
 [<ffffffff8118e058>] vfs_write+0xb8/0x1a0
 [<ffffffff8118eae2>] sys_pwrite64+0x82/0xa0
 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
Code: b3 cb e0 48 8b 95 48 ff ff ff 48 8b 42 08 44 8b 78 1c 45 85 ff 0f 88 dd 02 00 00 81 78 18 00 10 00 00 0f 87 c8 02 00 00 48 8b 10 <48> 3b 42 08 0f 85 bf 02 00 00 4d 85 e4 0f 85 10 fd ff ff 44 8b 
RIP  [<ffffffffa050cde5>] vx_dev_strategy+0x495/0x7c0 [vxfs]
 RSP <ffff881a7358e7d8>
CR2: 0000000000000008

Environment

  • Red Hat Enterprise Linux 6
    • seen on various kernels including 2.6.32-220.el6 and 2.6.32-504.8.1.el6,
  • vxfs

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content