"general protection fault" is reported when creating many namespaces/partitions on nvme disks

Issue

• "general protection fault" is reported when creating many namespaces/partitions on nvme disks

[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e020 flags=0x0020]
[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e028 flags=0x0020]
[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e030 flags=0x0020]
[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e038 flags=0x0020]
[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e040 flags=0x0020]
[Thu Aug 17 19:08:21 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb698e048 flags=0x0020]
[... ]
[Thu Aug 17 19:09:09 EDT 2023] nvme nvme3: rescanning namespaces.
[Thu Aug 17 19:09:09 EDT 2023]  nvme3n1:
[Thu Aug 17 19:09:09 EDT 2023]  nvme3n1:
[Thu Aug 17 19:09:09 EDT 2023]  nvme3n1:
[Thu Aug 17 19:09:09 EDT 2023]  nvme3n1: p1
[Thu Aug 17 19:09:11 EDT 2023]  nvme3n1: p1 p2
[Thu Aug 17 19:09:14 EDT 2023]  nvme3n1: p1 p2 p3 p4
[... ]
[Thu Aug 17 19:10:10 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb9a4f040 flags=0x0020]
[Thu Aug 17 19:10:10 EDT 2023] nvme 0000:81:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0045 address=0xb9a4f000 flags=0x0020]
[... ]
[Thu Aug 17 19:10:31 EDT 2023]  nvme3n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29 p30 p31 p32 p33 p34 p35 p36 p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 p47 p48 p49 p50 p51 p52 p53 p54 p55 p56 p57 p58 p59 p60 p61 p62 p63
[Thu Aug 17 19:10:31 EDT 2023]  nvme3n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29 p30 p31 p32 p33 p34 p35 p36 p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 p47 p48 p49 p50 p51 p52 p53 p54 p55 p56 p57 p58 p59 p60 p61 p62 p63
[Thu Aug 17 19:10:31 EDT 2023]  nvme3n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29 p30 p31 p32 p33 p34 p35 p36 p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 p47 p48 p49 p50 p51 p52 p53 p54 p55 p56 p57 p58 p59 p60 p61 p62 p63
[Thu Aug 17 19:10:32 EDT 2023] general protection fault, probably for non-canonical address 0x6f1f5c57218b6a63: 0000 [#1] PREEMPT SMP NOPTI
[Thu Aug 17 19:10:32 EDT 2023] CPU: 41 PID: 72419 Comm: systemd-udevd Kdump: loaded Not tainted 5.14.0-284.25.1.el9_2.x86_64 #1
[Thu Aug 17 19:10:32 EDT 2023] Hardware name: Supermicro AS -1125HS-TNR/H13DSH, BIOS 1.4 04/19/2023
[Thu Aug 17 19:10:32 EDT 2023] RIP: 0010:__kmalloc+0x116/0x360
[Thu Aug 17 19:10:32 EDT 2023] Code: 79 48 8b 51 08 48 8b 01 48 83 79 10 00 48 89 04 24 0f 84 a7 01 00 00 48 85 c0 0f 84 9e 01 00 00 8b 4d 28 48 8b 7d 00 48 01 c1 <48> 8b 19 48 89 ce 48 33 9d b8 00 00 00 48 8d 8a 00 20 00 00 48 0f
[Thu Aug 17 19:10:32 EDT 2023] RSP: 0018:ff670ef9c50f78a0 EFLAGS: 00010206
[Thu Aug 17 19:10:32 EDT 2023] RAX: 6f1f5c57218b6a43 RBX: 0000000000000c00 RCX: 6f1f5c57218b6a63
[Thu Aug 17 19:10:32 EDT 2023] RDX: 00000001a9a9a029 RSI: 0000000000000c00 RDI: 00000000000300a0
[Thu Aug 17 19:10:32 EDT 2023] RBP: ff46a55780034500 R08: 0000000000000000 R09: 0000000000001000
[Thu Aug 17 19:10:32 EDT 2023] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[Thu Aug 17 19:10:32 EDT 2023] R13: 0000000000000c00 R14: 0000000000000040 R15: ffffffff86f55ddb
[Thu Aug 17 19:10:32 EDT 2023] FS:  00007ff7f4ac6540(0000) GS:ff46a5668c640000(0000) knlGS:0000000000000000
[Thu Aug 17 19:10:32 EDT 2023] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Thu Aug 17 19:10:32 EDT 2023] CR2: 00007f3450b82004 CR3: 00000001c7b54002 CR4: 0000000000771ee0
[Thu Aug 17 19:10:32 EDT 2023] PKRU: 55555554
[Thu Aug 17 19:10:32 EDT 2023] Call Trace:
[Thu Aug 17 19:10:32 EDT 2023]  <TASK>
[Thu Aug 17 19:10:32 EDT 2023]  bio_integrity_prep+0xcb/0x260
[Thu Aug 17 19:10:32 EDT 2023]  ? do_mpage_readpage+0x282/0x680
[Thu Aug 17 19:10:32 EDT 2023]  blk_mq_submit_bio+0xb4/0x3f0
[Thu Aug 17 19:10:32 EDT 2023]  submit_bio_noacct_nocheck+0xcf/0x130
[Thu Aug 17 19:10:32 EDT 2023]  mpage_readahead+0x165/0x1d0
[Thu Aug 17 19:10:32 EDT 2023]  ? blkdev_write_begin+0x20/0x20
[Thu Aug 17 19:10:32 EDT 2023]  read_pages+0x60/0x380
[Thu Aug 17 19:10:32 EDT 2023]  ? xas_load+0x5/0xa0
[Thu Aug 17 19:10:32 EDT 2023]  page_cache_ra_unbounded+0x10c/0x180
[Thu Aug 17 19:10:32 EDT 2023]  force_page_cache_ra+0xc5/0xf0
[Thu Aug 17 19:10:32 EDT 2023]  filemap_get_pages+0xec/0x350
[Thu Aug 17 19:10:32 EDT 2023]  filemap_read+0xb9/0x320
[Thu Aug 17 19:10:32 EDT 2023]  ? __ia32_sys_getdents+0xc0/0x120
[Thu Aug 17 19:10:32 EDT 2023]  blkdev_read_iter+0xaf/0x170
[Thu Aug 17 19:10:32 EDT 2023]  new_sync_read+0xfc/0x180
[Thu Aug 17 19:10:32 EDT 2023]  vfs_read+0x13c/0x190
[Thu Aug 17 19:10:32 EDT 2023]  ksys_read+0x5f/0xe0
[Thu Aug 17 19:10:32 EDT 2023]  do_syscall_64+0x5c/0x90
[Thu Aug 17 19:10:32 EDT 2023]  ? blkdev_llseek+0x4c/0x60
[Thu Aug 17 19:10:32 EDT 2023]  ? syscall_exit_work+0x11a/0x150
[Thu Aug 17 19:10:32 EDT 2023]  ? syscall_exit_to_user_mode+0x12/0x30
[Thu Aug 17 19:10:32 EDT 2023]  ? do_syscall_64+0x69/0x90
[Thu Aug 17 19:10:32 EDT 2023]  ? syscall_exit_work+0x11a/0x150
[Thu Aug 17 19:10:32 EDT 2023]  ? syscall_exit_to_user_mode+0x12/0x30
[Thu Aug 17 19:10:32 EDT 2023]  ? do_syscall_64+0x69/0x90
[Thu Aug 17 19:10:32 EDT 2023]  ? sysvec_apic_timer_interrupt+0x3c/0x90
[Thu Aug 17 19:10:32 EDT 2023]  entry_SYSCALL_64_after_hwframe+0x63/0xcd
[Thu Aug 17 19:10:32 EDT 2023] RIP: 0033:0x7ff7f553eaf2
[Thu Aug 17 19:10:32 EDT 2023] Code: c0 e9 b2 fe ff ff 50 48 8d 3d ca 0c 08 00 e8 65 ea 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
[Thu Aug 17 19:10:32 EDT 2023] RSP: 002b:00007ffc722efae8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[Thu Aug 17 19:10:32 EDT 2023] RAX: ffffffffffffffda RBX: 000055ab8ad30398 RCX: 00007ff7f553eaf2
[Thu Aug 17 19:10:32 EDT 2023] RDX: 0000000000040000 RSI: 000055ab8ad303a8 RDI: 000000000000000f
[Thu Aug 17 19:10:32 EDT 2023] RBP: 000055ab8ab3ce50 R08: 0000000000000001 R09: 0000000000000000
[Thu Aug 17 19:10:32 EDT 2023] R10: 0000000020000000 R11: 0000000000000246 R12: 0000000ba46c0000
[Thu Aug 17 19:10:32 EDT 2023] R13: 0000000000040000 R14: 000055ab8ad30380 R15: 000055ab8ab3cea0
[Thu Aug 17 19:10:32 EDT 2023]  </TASK>

Backtrace of the panic task shows

crash> bt
PID: 72419    TASK: ff46a5672fe8c600  CPU: 41   COMMAND: "systemd-udevd"
 #0 [ff670ef9c50f7610] machine_kexec at ffffffff86a6c237
 #1 [ff670ef9c50f7668] __crash_kexec at ffffffff86bc3fea
 #2 [ff670ef9c50f7728] crash_kexec at ffffffff86bc51a8
 #3 [ff670ef9c50f7730] oops_end at ffffffff86a291db
 #4 [ff670ef9c50f7750] exc_general_protection at ffffffff87525c83
 #5 [ff670ef9c50f77f0] asm_exc_general_protection at ffffffff87600aa2
    [exception RIP: __kmalloc+278]
    RIP: ffffffff86d7d8b6  RSP: ff670ef9c50f78a0  RFLAGS: 00010206
    RAX: 6f1f5c57218b6a43  RBX: 0000000000000c00  RCX: 6f1f5c57218b6a63
    RDX: 00000001a9a9a029  RSI: 0000000000000c00  RDI: 00000000000300a0
    RBP: ff46a55780034500   R8: 0000000000000000   R9: 0000000000001000
    R10: 0000000000000000  R11: 0000000000000000  R12: 0000000000000000
    R13: 0000000000000c00  R14: 0000000000000040  R15: ffffffff86f55ddb
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #6 [ff670ef9c50f78e0] bio_integrity_prep at ffffffff86f55ddb
 #7 [ff670ef9c50f7928] blk_mq_submit_bio at ffffffff86f2c3f4
 #8 [ff670ef9c50f7968] submit_bio_noacct_nocheck at ffffffff86f1d93f
 #9 [ff670ef9c50f79a8] mpage_readahead at ffffffff86e17835
#10 [ff670ef9c50f7a78] read_pages at ffffffff86ce3520
#11 [ff670ef9c50f7ad8] page_cache_ra_unbounded at ffffffff86ce394c
#12 [ff670ef9c50f7b40] force_page_cache_ra at ffffffff86ce3d45
#13 [ff670ef9c50f7b68] filemap_get_pages at ffffffff86cd7b1c
#14 [ff670ef9c50f7c28] filemap_read at ffffffff86cd7e39
#15 [ff670ef9c50f7d20] blkdev_read_iter at ffffffff86f1654f
#16 [ff670ef9c50f7d50] new_sync_read at ffffffff86dc448c
#17 [ff670ef9c50f7de8] vfs_read at ffffffff86dc6ffc
#18 [ff670ef9c50f7e20] ksys_read at ffffffff86dc760f
#19 [ff670ef9c50f7e58] do_syscall_64 at ffffffff8752515c
#20 [ff670ef9c50f7f50] entry_SYSCALL_64_after_hwframe at ffffffff8760009b
    RIP: 00007ff7f553eaf2  RSP: 00007ffc722efae8  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 000055ab8ad30398  RCX: 00007ff7f553eaf2
    RDX: 0000000000040000  RSI: 000055ab8ad303a8  RDI: 000000000000000f
    RBP: 000055ab8ab3ce50   R8: 0000000000000001   R9: 0000000000000000
    R10: 0000000020000000  R11: 0000000000000246  R12: 0000000ba46c0000
    R13: 0000000000040000  R14: 000055ab8ad30380  R15: 000055ab8ab3cea0
    ORIG_RAX: 0000000000000000  CS: 0033  SS: 002b
crash>