RHEL6.8: kernel crash due to memory corruption in one process's fdtable

Solution Unverified - Updated -

Issue

  • system crashed with RIP find_next_zero_bit while opening a file
general protection fault: 0000 [#1] SMP 
last sysfs file: /sys/devices/system/node/node0/meminfo
CPU 14 
Modules linked in: iptable_filter ip_tables scdrv(U) tcp_diag inet_diag nfs lockd fscache auth_rpcgss nfs_acl ipv6 sunrpc microcode sg virtio_balloon virtio_console virtio_net i2c_piix4 i2c_core ext4 jbd2 mbcache virtio_blk virtio_scsi sr_mod cdrom virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]

Pid: 22330, comm: bauu9021 Not tainted 2.6.32-642.15.1.el6.x86_64 #1 Red Hat RHEV Hypervisor
RIP: 0010:[<ffffffff81299e28>]  [<ffffffff81299e28>] find_next_zero_bit+0x68/0xc0
RSP: 0018:ffff880fd8a33eb8  EFLAGS: 00010206
RAX: 000000006e69662f RBX: ffff880fba13cd00 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 000000006e69662f RDI: 000000006e69662f
RBP: ffff880fd8a33eb8 R08: 43462f656c63616e R09: 0000000000000000
Failed to find vfsmount for dentry 0x5c785e00 ('TMPv7SjydJ0c.RIP') in hash. Skipping !!!
R10: 0000000000000001 R11: 0000000000000001 R12: ffff8823dd82a200
R13: 0000000000000000 R14: ffff8823dd82a180 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff8800283c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffcec6bfff0 CR3: 0000000ec46b0000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process bauu9021 (pid: 22330, threadinfo ffff880fd8a30000, task ffff880feab7cab0)
Stack:
 ffff880fd8a33f08 ffffffff811ba244 00008000d8a33fd8 ffff8823dd82a188
<d> ffff880feab7cab0 0000003756822188 00000000ffffff9c ffff880ff5876000
<d> 0000000000008000 0000000000000000 ffff880fd8a33f18 ffffffff811ba332
Call Trace:
 [<ffffffff811ba244>] alloc_fd+0x84/0x160
 [<ffffffff811ba332>] get_unused_fd_flags+0x12/0x20
 [<ffffffff81196bb9>] do_sys_open+0x49/0x130
 [<ffffffff81196ce0>] sys_open+0x20/0x30
 [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
Failed to find vfsmount for dentry 0x5c7940c0 ('TMPKxgOatJ0c.RPT') in hash. Skipping !!!
Code: 48 83 ff 3f 77 55 48 c7 c2 ff ff ff ff 89 f9 48 d3 e2 48 09 d0 48 83 f8 ff 74 58 48 f7 d0 48 0f bc c0 4c 01 c8 c9 c3 0f 1f 40 00 <49> 8b 00 48 83 f8 ff 75 e7 49 83 c0 08 49 83 c1 40 48 83 ef 40 
RIP  [<ffffffff81299e28>] find_next_zero_bit+0x68/0xc0
 RSP <ffff880fd8a33eb8>
  • prior to the crash, there were thousands of "Failed to find vfsmount for dentry" messages similar to the following
Failed to find vfsmount for dentry 0x5800d0c0 ('TMPFpQPnqJ0c.RIP') in hash. Skipping !!!

Environment

  • Red Hat Enterprise Linux 6.8
    • seen on kernel-2.6.32-642.15.1.el6
  • 3rd party module 'scdrv'
crash> mod -t
NAME   TAINTS
scdrv  (U)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content