System on VMware ESX 6.X crashes called from VMCI module

Solution Verified - Updated -

Issue

  • System crashes in Virtual Machine Communication Interface (VMCI) with the following stack trace:

    • Red Hat Enterprise Linux 6:
    BUG: unable to handle kernel NULL pointer dereference at (null)
    IP: [<(null)>] (null)
    PGD 113884067 PUD 342e35067 PMD 0 
    Oops: 0010 [#1] SMP 
    last sysfs file: /sys/devices/virtual/dmi/id/product_name
    CPU 3 
    Modules linked in: nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc     vsock(U) uinput ppdev parport_pc parport sg microcode serio_raw vmware_balloon     vmci(U) i2c_piix4 i2c_core shpchp ext4 jbd2 mbcache sd_mod crc_t10dif vmxnet3     vmw_pvscsi sr_mod cdrom pata_acpi ata_generic ata_piix dm_mirror dm_region_hash     dm_log dm_mod [last unloaded: speedstep_lib]
    
    Pid: 0, comm: swapper Not tainted 2.6.32-573.3.1.el6.x86_64 #1 VMware, Inc.     VMware Virtual Platform/440BX Desktop Reference Platform
    RIP: 0010:[<0000000000000000>]  [<(null)>] (null)
    RSP: 0018:ffff880028383b18  EFLAGS: 00010046
    RAX: ffffffff8160e4c0 RBX: ffff88043a2e4ab0 RCX: 0001ca27d74eb1fa
    RDX: 0000000000000001 RSI: ffff88043a2e4ab0 RDI: ffff8800283959c0
    RBP: ffff880028383b40 R08: 0000000000000000 R09: ffff880028383ddf
    R10: 0000000000000002 R11: 0000000000000001 R12: ffff8800283959c0
    R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000003
    FS:  0000000000000000(0000) GS:ffff880028380000(0000) knlGS:0000000000000000
    CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
    CR2: 0000000000000000 CR3: 000000010063a000 CR4: 00000000001407e0
    DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
    Process swapper (pid: 0, threadinfo ffff88043a2ec000, task ffff88043a2e4ab0)
    Stack:
     ffffffff8105e646 ffff880028383b50 ffff88043a2e4ab0 ffff8800283959c0
    <d> 0000000000000000 ffff880028383b50 ffffffff8105e683 ffff880028383bc0
    <d> ffffffff810670c0 0000000000000000 0000000000000003 ffff880028383c70
    Call Trace:
     <IRQ> 
     [<ffffffff8105e646>] ? enqueue_task+0x66/0x80
     [<ffffffff8105e683>] activate_task+0x23/0x30
     [<ffffffff810670c0>] try_to_wake_up+0x1f0/0x3e0
     [<ffffffff810672c2>] default_wake_function+0x12/0x20
     [<ffffffff81059939>] __wake_up_common+0x59/0x90
     [<ffffffff8105e168>] __wake_up+0x48/0x70
     [<ffffffffa0148e40>] ? vmci_event_unsubscribe+0x114/0x154 [vmci]
     [<ffffffffa0148e64>] vmci_event_unsubscribe+0x138/0x154 [vmci]
     [<ffffffffa013f992>] VMCI_WaitOnEventInterruptible+0x7e/0xc0 [vmci]
     [<ffffffff810672b0>] ? default_wake_function+0x0/0x20
     [<ffffffff8107fe48>] ? irq_exit+0x48/0x90
     [<ffffffffa0148da8>] vmci_event_unsubscribe+0x7c/0x154 [vmci]
     [<ffffffffa017db60>] VMCISock_KernelDeregister+0x54c/0x704 [vsock]
     [<ffffffff8145b303>] __sk_free+0x23/0x170
     [<ffffffff8145b4d5>] sk_free+0x25/0x30
     [<ffffffffa0180411>] VSockVmciAllowDgram+0x1e11/0x2e44 [vsock]
     [<ffffffffa0145759>] ? VMCIHashTable_GetEntry+0xa5/0xb8 [vmci]
     [<ffffffffa0142130>] VMCIDatagram_InvokeGuestHandler+0xf0/0x1cc [vmci]
     [<ffffffffa0141ce7>] VMCI_ReadDatagramsFromPort+0xd3/0x158 [vmci]
     [<ffffffff810804e5>] tasklet_action+0xe5/0x120
     [<ffffffff8107ffd1>] __do_softirq+0xc1/0x1e0
     [<ffffffff810ed980>] ? handle_IRQ_event+0x60/0x170
     [<ffffffff8100c38c>] call_softirq+0x1c/0x30
     [<ffffffff8100fbd5>] do_softirq+0x65/0xa0
     [<ffffffff8107fe85>] irq_exit+0x85/0x90
     [<ffffffff815424e5>] do_IRQ+0x75/0xf0
     [<ffffffff8100ba53>] ret_from_intr+0x0/0x11
     <EOI> 
     [<ffffffff8104301b>] ? native_safe_halt+0xb/0x10
     [<ffffffff810167ed>] default_idle+0x4d/0xb0
     [<ffffffff81009fe6>] cpu_idle+0xb6/0x110
     [<ffffffff81531932>] start_secondary+0x2c0/0x316
    Code:  Bad RIP value.
    RIP  [<(null)>] (null)
     RSP <ffff880028383b18>
    CR2: 0000000000000000
    
    • Red Hat Enterprise Linux 7:
    [201414.815859] BUG: scheduling while atomic: swapper/0/0/0x00000100
    [201414.815893] Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs     fscache sg vmw_vsock_vmci_transport vsock coretemp crct10dif_pclmul     crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul     glue_helper ablk_helper cryptd vmw_balloon serio_raw pcspkr vmw_vmci shpchp     i2c_piix4 mperf nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc ext4 mbcache     jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_common     vmwgfx ttm drm ata_piix vmxnet3 libata vmw_pvscsi i2c_core dm_mirror     dm_region_hash dm_log dm_mod
    [201414.815926] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-123.el7.x86_64     #1
    [201414.815930] Hardware name: VMware, Inc. VMware Virtual Platform/440BX     Desktop Reference Platform, BIOS 6.00 09/30/2014
    [201414.815933]  ffff880c8c603ce0 a6bf6e556d593bc9 ffff880c8c603b30     ffffffff815e19ba
    [201414.815938]  ffff880c8c603b40 ffffffff815dbfe9 ffff880c8c603ba0     ffffffff815e718b
    [201414.815942]  ffffffff818c3fd8 0000000000014580 ffffffff818c3fd8     0000000000014580
    [201414.815946] Call Trace:
    [201414.815949]  <IRQ>  [<ffffffff815e19ba>] dump_stack+0x19/0x1b
    [201414.815961]  [<ffffffff815dbfe9>] __schedule_bug+0x4d/0x5b
    [201414.815966]  [<ffffffff815e718b>] __schedule+0x78b/0x790
    [201414.815970]  [<ffffffff815e71b9>] schedule+0x29/0x70
    [201414.815975]  [<ffffffff815e50b9>] schedule_timeout+0x209/0x2d0
    [201414.815981]  [<ffffffff8108ec58>] ? __wake_up_common+0x58/0x90
    [201414.815985]  [<ffffffff81090a24>] ? __wake_up+0x44/0x50
    [201414.815990]  [<ffffffff815e76e6>] wait_for_completion+0x116/0x170
    [201414.815995]  [<ffffffff81097700>] ? wake_up_state+0x20/0x20
    [201414.816001]  [<ffffffff810ff6a0>] ? __call_rcu+0x2c0/0x2c0
    [201414.816008]  [<ffffffff810826ab>] wait_rcu_gp+0x5b/0x80
    [201414.816012]  [<ffffffff81082630>] ? ftrace_raw_output_rcu_utilization+0x50/    0x50
    [201414.816016]  [<ffffffff8108ec58>] ? __wake_up_common+0x58/0x90
    [201414.816020]  [<ffffffff810fef5b>] synchronize_sched+0x3b/0x50
    [201414.816027]  [<ffffffffa031a9d0>] vmci_event_unsubscribe+0x70/0xb0     [vmw_vmci]
    [201414.816032]  [<ffffffffa032f611>] vmci_transport_destruct+0x21/0xe0     [vmw_vsock_vmci_transport]
    [201414.816037]  [<ffffffffa03a0867>] vsock_sk_destruct+0x17/0x60 [vsock]
    [201414.816044]  [<ffffffff814bb73f>] __sk_free+0x1f/0x170
    [201414.816048]  [<ffffffff814bb8b1>] sk_free+0x21/0x30
    [201414.816052]  [<ffffffffa03302bb>] vmci_transport_recv_stream_cb+0x1fb/0x2e0     [vmw_vsock_vmci_transport]
    [201414.816058]  [<ffffffff810bd9a4>] ? tick_program_event+0x24/0x30
    [201414.816063]  [<ffffffffa0319f1c>] vmci_datagram_invoke_guest_handler+0xbc/    0xf0 [vmw_vmci]
    [201414.816069]  [<ffffffffa031ae6f>] vmci_dispatch_dgs+0xcf/0x230 [vmw_vmci]
    [201414.816075]  [<ffffffff81066eae>] tasklet_action+0x6e/0x110
    

Environment

  • Red Hat Enterprise Linux (RHEL) 6, 7
  • VMware ESX Server 6.X

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In