System on VMware ESX 6.X crashes called from VMCI module

Issue

• System crashes in Virtual Machine Communication Interface (VMCI) with the following stack trace:

  • Red Hat Enterprise Linux 6:

  BUG: unable to handle kernel NULL pointer dereference at (null)
  IP: [<(null)>] (null)
  PGD 113884067 PUD 342e35067 PMD 0 
  Oops: 0010 [#1] SMP 
  last sysfs file: /sys/devices/virtual/dmi/id/product_name
  CPU 3 
  Modules linked in: nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc     vsock(U) uinput ppdev parport_pc parport sg microcode serio_raw vmware_balloon     vmci(U) i2c_piix4 i2c_core shpchp ext4 jbd2 mbcache sd_mod crc_t10dif vmxnet3     vmw_pvscsi sr_mod cdrom pata_acpi ata_generic ata_piix dm_mirror dm_region_hash     dm_log dm_mod [last unloaded: speedstep_lib]
  
  Pid: 0, comm: swapper Not tainted 2.6.32-573.3.1.el6.x86_64 #1 VMware, Inc.     VMware Virtual Platform/440BX Desktop Reference Platform
  RIP: 0010:[<0000000000000000>]  [<(null)>] (null)
  RSP: 0018:ffff880028383b18  EFLAGS: 00010046
  RAX: ffffffff8160e4c0 RBX: ffff88043a2e4ab0 RCX: 0001ca27d74eb1fa
  RDX: 0000000000000001 RSI: ffff88043a2e4ab0 RDI: ffff8800283959c0
  RBP: ffff880028383b40 R08: 0000000000000000 R09: ffff880028383ddf
  R10: 0000000000000002 R11: 0000000000000001 R12: ffff8800283959c0
  R13: 0000000000000001 R14: 0000000000000003 R15: 0000000000000003
  FS:  0000000000000000(0000) GS:ffff880028380000(0000) knlGS:0000000000000000
  CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  CR2: 0000000000000000 CR3: 000000010063a000 CR4: 00000000001407e0
  DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  Process swapper (pid: 0, threadinfo ffff88043a2ec000, task ffff88043a2e4ab0)
  Stack:
   ffffffff8105e646 ffff880028383b50 ffff88043a2e4ab0 ffff8800283959c0
  <d> 0000000000000000 ffff880028383b50 ffffffff8105e683 ffff880028383bc0
  <d> ffffffff810670c0 0000000000000000 0000000000000003 ffff880028383c70
  Call Trace:
   <IRQ> 
   [<ffffffff8105e646>] ? enqueue_task+0x66/0x80
   [<ffffffff8105e683>] activate_task+0x23/0x30
   [<ffffffff810670c0>] try_to_wake_up+0x1f0/0x3e0
   [<ffffffff810672c2>] default_wake_function+0x12/0x20
   [<ffffffff81059939>] __wake_up_common+0x59/0x90
   [<ffffffff8105e168>] __wake_up+0x48/0x70
   [<ffffffffa0148e40>] ? vmci_event_unsubscribe+0x114/0x154 [vmci]
   [<ffffffffa0148e64>] vmci_event_unsubscribe+0x138/0x154 [vmci]
   [<ffffffffa013f992>] VMCI_WaitOnEventInterruptible+0x7e/0xc0 [vmci]
   [<ffffffff810672b0>] ? default_wake_function+0x0/0x20
   [<ffffffff8107fe48>] ? irq_exit+0x48/0x90
   [<ffffffffa0148da8>] vmci_event_unsubscribe+0x7c/0x154 [vmci]
   [<ffffffffa017db60>] VMCISock_KernelDeregister+0x54c/0x704 [vsock]
   [<ffffffff8145b303>] __sk_free+0x23/0x170
   [<ffffffff8145b4d5>] sk_free+0x25/0x30
   [<ffffffffa0180411>] VSockVmciAllowDgram+0x1e11/0x2e44 [vsock]
   [<ffffffffa0145759>] ? VMCIHashTable_GetEntry+0xa5/0xb8 [vmci]
   [<ffffffffa0142130>] VMCIDatagram_InvokeGuestHandler+0xf0/0x1cc [vmci]
   [<ffffffffa0141ce7>] VMCI_ReadDatagramsFromPort+0xd3/0x158 [vmci]
   [<ffffffff810804e5>] tasklet_action+0xe5/0x120
   [<ffffffff8107ffd1>] __do_softirq+0xc1/0x1e0
   [<ffffffff810ed980>] ? handle_IRQ_event+0x60/0x170
   [<ffffffff8100c38c>] call_softirq+0x1c/0x30
   [<ffffffff8100fbd5>] do_softirq+0x65/0xa0
   [<ffffffff8107fe85>] irq_exit+0x85/0x90
   [<ffffffff815424e5>] do_IRQ+0x75/0xf0
   [<ffffffff8100ba53>] ret_from_intr+0x0/0x11
   <EOI> 
   [<ffffffff8104301b>] ? native_safe_halt+0xb/0x10
   [<ffffffff810167ed>] default_idle+0x4d/0xb0
   [<ffffffff81009fe6>] cpu_idle+0xb6/0x110
   [<ffffffff81531932>] start_secondary+0x2c0/0x316
  Code:  Bad RIP value.
  RIP  [<(null)>] (null)
   RSP <ffff880028383b18>
  CR2: 0000000000000000
  

  • Red Hat Enterprise Linux 7:

  [201414.815859] BUG: scheduling while atomic: swapper/0/0/0x00000100
  [201414.815893] Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs     fscache sg vmw_vsock_vmci_transport vsock coretemp crct10dif_pclmul     crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul     glue_helper ablk_helper cryptd vmw_balloon serio_raw pcspkr vmw_vmci shpchp     i2c_piix4 mperf nfsd auth_rpcgss nfs_acl lockd sunrpc binfmt_misc ext4 mbcache     jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_common     vmwgfx ttm drm ata_piix vmxnet3 libata vmw_pvscsi i2c_core dm_mirror     dm_region_hash dm_log dm_mod
  [201414.815926] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.10.0-123.el7.x86_64     #1
  [201414.815930] Hardware name: VMware, Inc. VMware Virtual Platform/440BX     Desktop Reference Platform, BIOS 6.00 09/30/2014
  [201414.815933]  ffff880c8c603ce0 a6bf6e556d593bc9 ffff880c8c603b30     ffffffff815e19ba
  [201414.815938]  ffff880c8c603b40 ffffffff815dbfe9 ffff880c8c603ba0     ffffffff815e718b
  [201414.815942]  ffffffff818c3fd8 0000000000014580 ffffffff818c3fd8     0000000000014580
  [201414.815946] Call Trace:
  [201414.815949]  <IRQ>  [<ffffffff815e19ba>] dump_stack+0x19/0x1b
  [201414.815961]  [<ffffffff815dbfe9>] __schedule_bug+0x4d/0x5b
  [201414.815966]  [<ffffffff815e718b>] __schedule+0x78b/0x790
  [201414.815970]  [<ffffffff815e71b9>] schedule+0x29/0x70
  [201414.815975]  [<ffffffff815e50b9>] schedule_timeout+0x209/0x2d0
  [201414.815981]  [<ffffffff8108ec58>] ? __wake_up_common+0x58/0x90
  [201414.815985]  [<ffffffff81090a24>] ? __wake_up+0x44/0x50
  [201414.815990]  [<ffffffff815e76e6>] wait_for_completion+0x116/0x170
  [201414.815995]  [<ffffffff81097700>] ? wake_up_state+0x20/0x20
  [201414.816001]  [<ffffffff810ff6a0>] ? __call_rcu+0x2c0/0x2c0
  [201414.816008]  [<ffffffff810826ab>] wait_rcu_gp+0x5b/0x80
  [201414.816012]  [<ffffffff81082630>] ? ftrace_raw_output_rcu_utilization+0x50/    0x50
  [201414.816016]  [<ffffffff8108ec58>] ? __wake_up_common+0x58/0x90
  [201414.816020]  [<ffffffff810fef5b>] synchronize_sched+0x3b/0x50
  [201414.816027]  [<ffffffffa031a9d0>] vmci_event_unsubscribe+0x70/0xb0     [vmw_vmci]
  [201414.816032]  [<ffffffffa032f611>] vmci_transport_destruct+0x21/0xe0     [vmw_vsock_vmci_transport]
  [201414.816037]  [<ffffffffa03a0867>] vsock_sk_destruct+0x17/0x60 [vsock]
  [201414.816044]  [<ffffffff814bb73f>] __sk_free+0x1f/0x170
  [201414.816048]  [<ffffffff814bb8b1>] sk_free+0x21/0x30
  [201414.816052]  [<ffffffffa03302bb>] vmci_transport_recv_stream_cb+0x1fb/0x2e0     [vmw_vsock_vmci_transport]
  [201414.816058]  [<ffffffff810bd9a4>] ? tick_program_event+0x24/0x30
  [201414.816063]  [<ffffffffa0319f1c>] vmci_datagram_invoke_guest_handler+0xbc/    0xf0 [vmw_vmci]
  [201414.816069]  [<ffffffffa031ae6f>] vmci_dispatch_dgs+0xcf/0x230 [vmw_vmci]
  [201414.816075]  [<ffffffff81066eae>] tasklet_action+0x6e/0x110