Kernel panic while running containerized DPDK application “kernel BUG at mm/memory.c:2102!“

Issue

• Encountering the following Kernel Panic when running a containerized DPDK application. The application sends and receives packets over VFs created on a 10G NIC (very low traffic rate).

[ 1806.630592] vfio-pci 0000:86:02.0: enabling device (0000 -> 0002)
[ 1808.060706] i40e 0000:86:00.0: Unprivileged VF 0 is attempting to configure promiscuous mode
[ 1816.397005] vfio-pci 0000:86:02.1: enabling device (0000 -> 0002)
[ 1817.673483] i40e 0000:86:00.0: Unprivileged VF 1 is attempting to configure promiscuous mode
[ 1927.667627] vfio-pci 0000:86:02.0: enabling device (0000 -> 0002)
[ 1929.100719] i40e 0000:86:00.0: Unprivileged VF 0 is attempting to configure promiscuous mode
[ 2000.534237] ------------[ cut here ]------------
[ 2000.534238] kernel BUG at mm/memory.c:2102!
[ 2000.534246] invalid opcode: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 2000.534249] CPU: 4 PID: 70055 Comm: lcore-slave-4 Kdump: loaded Tainted: G          I      --------- -  - 4.18.0-240.22.1.rt7.77.el8_3.x86_64 #1
[ 2000.534250] Hardware name: Intel Corporation S2600WF0/S2600WF0, BIOS SE5C620.86B.02.01.0009.092820190230 09/28/2019
[ 2000.534257] RIP: 0010:remap_pfn_range+0x48e/0x5c0
[ 2000.534259] Code: 85 f6 0f 85 78 ff ff ff 48 21 f9 31 ff eb 92 48 81 e1 00 00 00 c0 e9 42 fe ff ff 48 8b 44 24 50 48 89 44 24 40 e9 0e fd ff ff <0f> 0b 48 81 e3 00 f0 ff ff e9 1e ff ff ff bb f3 ff ff ff 0f 1f 80
[ 2000.534260] RSP: 0000:ffffb0bc2e8cfca8 EFLAGS: 00010286
[ 2000.534263] RAX: 0000000000000000 RBX: ffff8ae8d7fdb000 RCX: 0000000000000001
[ 2000.534264] RDX: 0000000000000001 RSI: ffffffffa12cf7c3 RDI: 00000000ffffffff
[ 2000.534266] RBP: 00000000000d5e00 R08: ffffd1605f602f88 R09: 00000000000336c7
[ 2000.534267] R10: 0000000000033670 R11: 0000000000033602 R12: ffff8af8ccce0000
[ 2000.534269] R13: 0000004300010000 R14: 8000000000000037 R15: 0000004300000000
[ 2000.534271] FS:  00007ff619143700(0000) GS:ffff8af1bf700000(0000) knlGS:0000000000000000
[ 2000.534272] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2000.534273] CR2: 0000004300000000 CR3: 0000002000196001 CR4: 00000000007606a0
[ 2000.534274] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2000.534275] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 2000.534276] PKRU: 55555554
[ 2000.534278] Call Trace:
[ 2000.534288]  vfio_pci_mmap_fault+0xe1/0x130 [vfio_pci]
[ 2000.534292]  __do_fault+0x38/0xd0
[ 2000.534295]  __handle_mm_fault+0x6f0/0xa70
[ 2000.534299]  handle_mm_fault+0xd2/0x1e0
[ 2000.534303]  __do_page_fault+0x28e/0x5d0
[ 2000.534306]  do_page_fault+0x47/0x1b0
[ 2000.534311]  ? page_fault+0x8/0x30
[ 2000.534313]  page_fault+0x1e/0x30
[ 2000.534316] RIP: 0033:0x86932f
[ 2000.534317] Code: 41 03 57 38 48 c1 e1 04 49 03 4f 10 48 8b 71 08 48 83 ce 20 48 89 71 08 66 41 89 57 52 66 45 89 67 20 49 8b 57 28 45 0f b7 e4 <44> 89 22 48 81 c4 80 02 00 00 5b 41 5a 41 5c 41 5d 41 5e 41 5f 5d
[ 2000.534318] RSP: 002b:00007ff619141260 EFLAGS: 00010297