Kernel panics due to a stack overrun with vxodm/vxfs task data

Solution Unverified - Updated -

Issue

  • Kernel panics due to a stack overrun with vxodm/vxfs task data
BUG: scheduling while atomic: oracle/40855/0xffff8810
BUG: unable to handle kernel NULL pointer dereference at 0000000000000178
IP: [<ffffffff810549df>] check_preempt_wakeup+0x17f/0x260
PGD 640bc0067 PUD 640bbd067 PMD 0 
Oops: 0000 [#1] SMP 
last sysfs file: /sys/devices/system/node/node3/meminfo
CPU 0 
Modules linked in: xfs ext3 jbd ext2 bridge nfs lockd fscache nfs_acl auth_rpcgss vxodm(P)(U) amf(P)(U) vxfen(P)(U) gab(P)(U) llt(P)(U) autofs4 sunrpc dmpjbod(P)(U) dmpap(P)(U) dmpaa(P)(U) vxspec(P)(U) vxio(P)(U) vxdmp(P)(U) bonding 8021q garp stp llc ipv6 vxportal(P)(U) fdd(P)(U) vxfs(P)(U) exportfs dcdbas microcode sb_edac edac_core iTCO_wdt iTCO_vendor_support shpchp ioatdma acpi_pad power_meter sg ixgbe dca bnx2x libcrc32c mdio ext4 mbcache jbd2 sd_mod crc_t10dif megaraid_sas wmi lpfc scsi_transport_fc scsi_tgt dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]

Pid: 1908, comm: gzip Tainted: P        W  ---------------    2.6.32-279.el6.x86_64 #1 Dell Inc. PowerEdge M820/034PY5
RIP: 0010:[<ffffffff810549df>]  [<ffffffff810549df>] check_preempt_wakeup+0x17f/0x260
RSP: 0000:ffff8800282034e0  EFLAGS: 00010087
RAX: ffff88105c7166e8 RBX: ffff880028216680 RCX: 0000000000000000
RDX: 0000000000000001 RSI: ffff8807f82ce080 RDI: ffff8800282166e8
RBP: ffff880028203530 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: ffff88201d1c3500 R15: 0000000000000000
FS:  00007f0fc783d700(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000178 CR3: 00000006521e0000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process gzip (pid: 1908, threadinfo ffff8800a1960000, task ffff88201d1c3500)
Stack:
 ffffffff81faff00 0000000000000000 ffffffff81faff00 0000000025736548
<d> ffff880028203540 ffff880028216680 ffff880028216680 0000000000000000
<d> 0000000000000000 0000000000000003 ffff880028203550 ffffffff8104f2cc
Call Trace:
 <IRQ> 
 [<ffffffff8104f2cc>] check_preempt_curr+0x7c/0x90
 [<ffffffff81060083>] try_to_wake_up+0x213/0x3e0
 [<ffffffff81060262>] default_wake_function+0x12/0x20
 [<ffffffff810920e6>] autoremove_wake_function+0x16/0x40
 [<ffffffff8104e309>] __wake_up_common+0x59/0x90
 [<ffffffff810533e8>] __wake_up+0x48/0x70
 [<ffffffff811168cd>] mempool_free+0x6d/0xa0
 [<ffffffff811b2634>] bio_free+0x64/0x70
 [<ffffffff811b2655>] bio_fs_destructor+0x15/0x20
 [<ffffffff811b13fb>] bio_put+0x2b/0x40
 [<ffffffffa04d830b>] vx_end_io+0x1b/0x50 [vxfs]
 [<ffffffffa04d8359>] vx_end_io_v2+0x19/0x20 [vxfs]
 [<ffffffff811b12ed>] bio_endio+0x1d/0x40
 [<ffffffff81254cdb>] req_bio_endio+0x9b/0xe0
 [<ffffffff81256737>] blk_update_request+0x107/0x490
 [<ffffffff81256ae7>] blk_update_bidi_request+0x27/0xa0
 [<ffffffff81257b8e>] __blk_end_request_all+0x2e/0x60
 [<ffffffffa0843b96>] vxvm_end_request+0x46/0x70 [vxio]
 [<ffffffffa08b411e>] volkiodone+0x12e/0x340 [vxio]
 [<ffffffffa0074063>] ? lpfc_sli_handle_fast_ring_event+0x2a3/0x770 [lpfc]
 [<ffffffff8102a869>] ? native_send_call_func_single_ipi+0x39/0x40
 [<ffffffff810a83b1>] ? generic_exec_single+0xb1/0xc0
 [<ffffffff810305de>] ? physflat_send_IPI_mask+0xe/0x10
 [<ffffffff8102a869>] ? native_send_call_func_single_ipi+0x39/0x40
 [<ffffffff810a83b1>] ? generic_exec_single+0xb1/0xc0
 [<ffffffff812718f9>] ? cpumask_next_and+0x29/0x50
 [<ffffffffa092e61f>] ? vol_free+0xef/0x100 [vxio]
 [<ffffffffa08e48f7>] volsiodone+0x457/0x710 [vxio]
 [<ffffffffa08e875e>] vol_subdisksio_done+0x22e/0x390 [vxio]
 [<ffffffffa08b15c9>] volkcontext_process+0x79/0x1c0 [vxio]
 [<ffffffffa0890b41>] voldiskiodone+0x2e1/0x3e0 [vxio]
 [<ffffffffa00a94b2>] ? lpfc_release_scsi_buf_s3+0x62/0x80 [lpfc]
 [<ffffffff81012bd9>] ? read_tsc+0x9/0x20
 [<ffffffffa0893192>] voldiskiodone_intr+0xd2/0x110 [vxio]
 [<ffffffffa0841766>] voldmp_iodone+0x16/0x20 [vxio]
 [<ffffffff811b12ed>] bio_endio+0x1d/0x40
 [<ffffffffa07685ca>] gendmpiodone+0x1da/0x270 [vxdmp]
 [<ffffffffa0769a12>] dmpiodone+0x52/0xf0 [vxdmp]
 [<ffffffff811b12ed>] bio_endio+0x1d/0x40
 [<ffffffff81254cdb>] req_bio_endio+0x9b/0xe0
 [<ffffffff81256737>] blk_update_request+0x107/0x490
 [<ffffffff81256ae7>] blk_update_bidi_request+0x27/0xa0
 [<ffffffff81257f2f>] blk_end_bidi_request+0x2f/0x80
 [<ffffffff81257fd0>] blk_end_request+0x10/0x20
 [<ffffffff8136c00f>] scsi_io_completion+0xaf/0x6c0
 [<ffffffff81363112>] scsi_finish_command+0xc2/0x130
 [<ffffffff8136c785>] scsi_softirq_done+0x145/0x170
 [<ffffffff8125d605>] blk_done_softirq+0x85/0xa0
 [<ffffffff810db800>] ? handle_IRQ_event+0x60/0x170
 [<ffffffff81073ec1>] __do_softirq+0xc1/0x1e0
 [<ffffffff8100c24c>] call_softirq+0x1c/0x30
 [<ffffffff8100de85>] do_softirq+0x65/0xa0
 [<ffffffff81073ca5>] irq_exit+0x85/0x90
 [<ffffffff8102a905>] smp_call_function_single_interrupt+0x35/0x40
 [<ffffffff8100bdb3>] call_function_single_interrupt+0x13/0x20
 <EOI> 
 [<ffffffff8150011c>] ? _spin_lock+0x1c/0x30
 [<ffffffff8114934e>] page_referenced+0x9e/0x370
 [<ffffffff8112d60d>] shrink_page_list.clone.0+0x17d/0x660
 [<ffffffff8112de0c>] shrink_inactive_list+0x31c/0x7d0
 [<ffffffff8112e727>] ? shrink_active_list+0x287/0x360
 [<ffffffff811287aa>] ? determine_dirtyable_memory+0x1a/0x30
 [<ffffffff81128857>] ? get_dirty_limits+0x27/0x2f0
 [<ffffffff8112eb8f>] shrink_zone+0x38f/0x520
 [<ffffffff8109cd39>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff8112ee1e>] do_try_to_free_pages+0xfe/0x520
 [<ffffffff8112f42d>] try_to_free_pages+0x9d/0x130
 [<ffffffff81130580>] ? isolate_pages_global+0x0/0x350
 [<ffffffff8112722d>] __alloc_pages_nodemask+0x40d/0x940
 [<ffffffff8115c1da>] alloc_pages_current+0xaa/0x110
 [<ffffffff811144e7>] __page_cache_alloc+0x87/0x90
 [<ffffffff811152ef>] find_or_create_page+0x4f/0xb0
 [<ffffffffa04d06f5>] vx_segmap_getmap+0x125/0xa00 [vxfs]
 [<ffffffff81010bce>] ? __copy_from_user_inatomic+0xe/0x20
 [<ffffffffa0427484>] ? vx_rwsleep_rec_unlock+0x54/0x110 [vxfs]
 [<ffffffffa0504899>] vx_write1_fast+0x349/0xe00 [vxfs]
 [<ffffffffa05060a0>] vx_write_common_fast+0x40/0x150 [vxfs]
 [<ffffffffa05065da>] vx_write_common+0x42a/0x790 [vxfs]
 [<ffffffffa048dc7c>] vx_write+0x1cc/0x290 [vxfs]
 [<ffffffff8117b068>] vfs_write+0xb8/0x1a0
 [<ffffffff810d69e2>] ? audit_syscall_entry+0x272/0x2a0
 [<ffffffff8117ba81>] sys_write+0x51/0x90
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Code: 7f f2 49 8b 85 78 01 00 00 49 39 84 24 78 01 00 00 74 29 66 0f 1f 84 00 00 00 00 00 4d 8b a4 24 70 01 00 00 4d 8b ad 70 01 00 00 <49> 8b 85 78 01 00 00 49 39 84 24 78 01 00 00 75 e0 4d 85 ed 0f 
RIP  [<ffffffff810549df>] check_preempt_wakeup+0x17f/0x260
 RSP <ffff8800282034e0>
CR2: 0000000000000178

Environment

  • Red Hat Enterprise Linux 6.3 (kernel-2.6.32-279.el6.x86_64)
  • Symantec Veritas VxFS (VRTSvxfs-6.0.100.000-GA_RHEL6.x86_64)
  • Oracle Database 12c

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.