The kernel crashes due to a GPF occurs in hv_pci_remove(). A possible race between hv_pci_eject_device()/hv_pci_eject_device() and hv_pci_remove().

Solution Unverified - Updated -

Issue

  • The kernel crashes due to a GPF occurs in hv_pci_remove().
[5029032.184638] general protection fault: 0000 [#1] SMP 
[5029032.185074] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver iptable_nat nf_nat_ipv4 nf_nat ip6_tables iptable_filter nf_conntrack_ipv4 nf_defrag_ipv4 xt_owner iptable_security xt_conntrack nf_conntrack nfsv3 nfs_acl nfs lockd grace fscache falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10803(E) sunrpc dm_mirror dm_region_hash dm_log mlx5_ib ib_uverbs ib_core mlx5_core mlxfw devlink joydev iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw dm_mod gf128mul glue_helper ablk_helper cryptd pcspkr i2c_piix4 hv_utils sg ptp pci_hyperv hv_balloon pps_core binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi hv_storvsc scsi_transport_fc hv_netvsc hid_hyperv hyperv_keyboard scsi_tgt
[5029032.185074]  ata_piix crct10dif_pclmul hyperv_fb crct10dif_common libata crc32c_intel hv_vmbus floppy serio_raw
[5029032.508663] CPU: 12 PID: 26482 Comm: kworker/12:2 Kdump: loaded Tainted: P            E  ------------   3.10.0-1160.11.1.el7.x86_64 #1
[5029032.508663] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
[5029032.508663] Workqueue: events vmbus_onmessage_work [hv_vmbus]
[5029032.508663] task: ffff953679fed280 ti: ffff953356b54000 task.ti: ffff953356b54000
[5029032.508663] RIP: 0010:[<ffffffffc07af2e8>]  [<ffffffffc07af2e8>] hv_pci_remove+0xd8/0x120 [pci_hyperv]
[5029032.508663] RSP: 0000:ffff953356b57ca8  EFLAGS: 00010287
[5029032.508663] RAX: 0000000000000000 RBX: ffff953609fbf000 RCX: 00000000000231a1
[5029032.508663] RDX: 0000000000000001 RSI: ffffd87753cafb00 RDI: ffff953356b57c40
[5029032.508663] RBP: ffff953356b57cc8 R08: 000000000001f040 R09: ffffffff8d788c0b
[5029032.508663] R10: ffff95b05f91f040 R11: ffffd87753ffe680 R12: dead000000000100
[5029032.508663] R13: ffff953678fe6400 R14: ffff953609fbf0b0 R15: 0000000000000300
[5029032.508663] FS:  0000000000000000(0000) GS:ffff95b05f900000(0000) knlGS:0000000000000000
[5029032.508663] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[5029032.508663] CR2: 00000000e8f38000 CR3: 00000080fd754000 CR4: 00000000003606e0
[5029032.508663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[5029032.508663] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[5029032.508663] Call Trace:
[5029032.508663]  [<ffffffffc005f02a>] vmbus_remove+0x2a/0x40 [hv_vmbus]
[5029032.508663]  [<ffffffff8d8bae12>] __device_release_driver+0x82/0xf0
[5029032.508663]  [<ffffffff8d8baea3>] device_release_driver+0x23/0x30
[5029032.508663]  [<ffffffff8d8ba56b>] bus_remove_device+0x10b/0x180
[5029032.508663]  [<ffffffff8d8b65ca>] device_del+0x13a/0x210
[5029032.508663]  [<ffffffff8d8b66be>] device_unregister+0x1e/0x60
[5029032.508663]  [<ffffffffc006166b>] vmbus_device_unregister+0x2b/0x60 [hv_vmbus]
[5029032.508663]  [<ffffffffc00651da>] vmbus_onoffer_rescind+0x15a/0x1b0 [hv_vmbus]
[5029032.508663]  [<ffffffffc00654b8>] vmbus_onmessage+0x48/0xf0 [hv_vmbus]
[5029032.508663]  [<ffffffff8db868ef>] ? __schedule+0x3af/0x860
[5029032.508663]  [<ffffffffc0060ab1>] vmbus_onmessage_work+0x21/0x30 [hv_vmbus]
[5029032.508663]  [<ffffffff8d4bde9f>] process_one_work+0x17f/0x440
[5029032.508663]  [<ffffffff8d4befb6>] worker_thread+0x126/0x3c0
[5029032.508663]  [<ffffffff8d4bee90>] ? manage_workers.isra.26+0x2a0/0x2a0
[5029032.508663]  [<ffffffff8d4c5e71>] kthread+0xd1/0xe0
[5029032.508663]  [<ffffffff8d4c5da0>] ? insert_kthread_work+0x40/0x40
[5029032.508663]  [<ffffffff8db93df7>] ret_from_fork_nospec_begin+0x21/0x21
[5029032.508663]  [<ffffffff8d4c5da0>] ? insert_kthread_work+0x40/0x40
[5029032.508663] Code: 00 00 00 e8 1b c4 01 cd 48 8b bb 88 00 00 00 4c 8d b3 b0 00 00 00 e8 d8 03 02 cd 4c 8b a3 b0 00 00 00 4d 39 f4 74 25 0f 1f 40 00 <49> 8b 7c 24 18 48 85 ff 74 0e e8 c9 13 03 cd 49 c7 44 24 18 00 
[5029032.508663] RIP  [<ffffffffc07af2e8>] hv_pci_remove+0xd8/0x120 [pci_hyperv]
[5029032.508663]  RSP <ffff953356b57ca8>
[5029033.578029] ---[ end trace 7705560df88f3d5a ]---
[5029033.592144] Kernel panic - not syncing: Fatal exception

Environment

  • Red Hat Enterprise Linux 7 (rhel7.9.z kernel-3.10.0-1160.35.1.el7 and backwards)
  • Red Hat Enterprise Linux 8 (rhel8.4.z and backwards)
  • MS Hyper-V guest

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content