After upgrading the kernel, ice/iavf driver and SR-IOV workloads stop working
Issue
- After upgrading the kernel, the SR-IOV workloads stop functioning and hit the following error in the guest:
iavf 0000:00:04.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
- Network flows are interrupted, the error keeps repeating.
- Panic occurred with below logs:
[ 2274.923162] iavf 0000:00:04.0 eth0: NIC Link is Up Speed is 25 Gbps Full Duplex
[ 2279.135890] iavf 0000:00:04.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
[ 2279.156168] iavf 0000:00:04.0 eth0: NIC Link is Up Speed is 25 Gbps Full Duplex
[ 2285.062016] BUG: unable to handle kernel paging request at 000054860000343f
[ 2285.063051] PGD 0
[ 2285.063419] Oops: 0002 [#1] SMP NOPTI
[ 2285.063964] CPU: 3 PID: 4861 Comm: kworker/u8:1 Kdump: loaded Tainted: G W --------- - - 4.18.0-425.3.1.el8.x86_64 #1
[ 2285.065586] Hardware name: Red Hat OpenStack Compute, BIOS 1.14.0-1.module+el8.4.0+8855+a9e237a9 04/01/2014
[ 2285.066863] Workqueue: iavf iavf_reset_task [iavf]
[ 2285.067557] RIP: 0010:__page_frag_cache_drain+0x5/0x30
[ 2285.068284] Code: f6 c4 80 74 0f 0f b6 77 51 85 f6 74 07 31 d2 e9 d1 e0 ff ff e9 2c ff ff ff 48 8b 05 05 42 52 01 eb b0 0f 1f 00 0f 1f 44 00 00 <f0> 29 77 34 74 05 e9 60 d6 90 00 48 8b 07 f6 c4 80 74 0f 0f b6 77
[ 2285.070735] RSP: 0018:ff4c815a0624fdf0 EFLAGS: 00010292
[ 2285.071474] RAX: ffffffff9d5b9740 RBX: ff20f48b074b0000 RCX: 0000000000000002
[ 2285.072441] RDX: 00000000af000000 RSI: 0000000000000000 RDI: 000054860000340b
[ 2285.073412] RBP: ff20f48b0a5d1c00 R08: 0000000000000022 R09: ff20f48b004027e0
[ 2285.074384] R10: 587829f300000000 R11: 0000000000000020 R12: 0000000000000000
[ 2285.075357] R13: 0000000000001000 R14: ff20f48b1dbc0d10 R15: 0000000000001760
[ 2285.076342] FS: 0000000000000000(0000) GS:ff20f48b3bb80000(0000) knlGS:0000000000000000
[ 2285.077523] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2285.078339] CR2: 000054860000343f CR3: 000000011dc10004 CR4: 0000000000771ee0
[ 2285.079316] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2285.080295] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 2285.081272] PKRU: 55555554
[ 2285.081724] Call Trace:
[ 2285.082141] iavf_clean_rx_ring+0xad/0x120 [iavf]
[ 2285.082836] iavf_free_rx_resources+0xe/0x50 [iavf]
[ 2285.083551] iavf_free_all_rx_resources.part.52+0x30/0x50 [iavf]
[ 2285.084393] iavf_reset_task+0x1b3/0x800 [iavf]
[ 2285.085061] process_one_work+0x1a7/0x360
[ 2285.085667] ? create_worker+0x1a0/0x1a0
[ 2285.086258] worker_thread+0x30/0x390
[ 2285.086818] ? create_worker+0x1a0/0x1a0
[ 2285.087401] kthread+0x10b/0x130
[ 2285.087910] ? set_kthread_struct+0x50/0x50
[ 2285.088530] ret_from_fork+0x1f/0x40
[ 2285.089079] Modules linked in: nls_utf8 isofs intel_rapl_msr intel_rapl_common nfit libnvdimm cirrus kvm_intel drm_shmem_helper drm_kms_helper kvm syscopyarea sysfillrect irqbypass sysimgblt fb_sys_fops drm pcspkr joydev i2c_piix4 virtio_balloon vfat fat xfs libcrc32c sr_mod cdrom sg ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel libata iavf ghash_clmulni_intel serio_raw virtio_blk sunrpc dm_mirror dm_region_hash dm_log dm_mod
[ 2285.094588] CR2: 000054860000343f
Environment
- Red Hat OpenStack Platform 16.2
- Red Hat Enterprise Linux 8.4 EUS/TUS
- kernel-4.18.0-305.49.1.el8_4
- kernel-4.18.0-305.97.1.el8_4
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.