The kernel crashes repeatedly on boot due to an OOB bug in the kernel's inbox mpt3sas driver

Issue

• The kernel crashes repeatedly on boot due to an OOB bug in the kernel's inbox mpt3sas driver.

        ...
megaraid_sas 0000:61:00.0: scanning for scsi0...
megaraid_sas 0000:61:00.0: 20047 (792829341s/0x0001/CRIT) - VD ee/1 is now DEGRADED
megaraid_sas 0000:61:00.0: scanning for scsi0...
systemd-udevd[34358]: segfault at 7f8a71f2a586 ip 00007f8a70ff8e35 sp 00007ffd84a287d8 error 5 in libc-2.28.so[7f8a70f2d000+1bb000]
Code: 00 00 0f 1f 00 31 c0 c5 f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f9 48 89 fa c5 f9 ef c0 83 e1 3f 83 f9 20 77 2b <c5> fd 74 0f c5 fd d7 c1 85 c0 0f 85 eb 00 00 00 48 83 c7 20 83 e1
swap_info_get: Bad swap offset entry 3fffffffffc67
BUG: Bad page map in process systemd-udevd  pte:00073000 pmd:a898b65067
addr:000000003128fe43 vm_flags:08000075 anon_vma:0000000000000000 mapping:00000000e2788aaa index:62
file:libsystemd-shared-239.so fault:xfs_filemap_fault [xfs] mmap:xfs_file_mmap [xfs] readpage:xfs_vm_readpage [xfs]
CPU: 184 PID: 34358 Comm: systemd-udevd Kdump: loaded Not tainted 4.18.0-477.89.1.el8_8.x86_64 #1
   ...
Call Trace:
 dump_stack+0x41/0x60
 print_bad_pte.cold.110+0x63/0xa6
 unmap_page_range+0x98b/0xf40
 unmap_vmas+0xc0/0xe0
 exit_mmap+0x9d/0x170
 mmput+0x58/0x130
 do_exit+0x2fb/0xb00
 do_group_exit+0x3a/0xa0
 get_signal+0x158/0x870
 ? __send_signal+0x359/0x4b0
 ? page_fault+0x8/0x30
 do_signal+0x36/0x690
 ? force_sig_info+0xc7/0xe0
 ? srso_alias_return_thunk+0x5/0xfcdfd
 ? force_sig_fault+0x59/0x80
 ? page_fault+0x8/0x30
 exit_to_usermode_loop+0x89/0x100
 prepare_exit_to_usermode+0x9f/0xb0
 retint_user+0x8/0x8
RIP: 0033:0x7f8a70ff8e35
Code: Unable to access opcode bytes at RIP 0x7f8a70ff8e0b.
RSP: 002b:00007ffd84a287d8 EFLAGS: 00010283
RAX: 00007f8a72232a20 RBX: 00007f8a71f2a586 RCX: 0000000000000006
RDX: 00007f8a71f2a586 RSI: 00007ffd84a28800 RDI: 00007f8a71f2a586
RBP: 00007ffd84a28800 R08: 12229ca7e648c4d4 R09: 43535f5952544e45
R10: 0000000000000019 R11: 9cd10eaf3d974cca R12: 00007f8a71f2a586
R13: 000055a1c3af265c R14: 000055a1c57ed0a0 R15: 000055a1c5833c00
Disabling lock debugging due to kernel taint
        ...
swap_info_get: Bad swap offset entry 3fffffffffca8
swap_info_get: Bad swap file entry 402cf6338af330e
swap_info_get: Bad swap offset entry 3fffffffffc50
swap_info_get: Bad swap file entry c009dbafde899de
swap_info_get: Bad swap file entry 400f7c312cdd0bb
BUG: Bad rss-counter state mm:000000002808156e idx:0 val:238
BUG: Bad rss-counter state mm:000000002808156e idx:2 val:-181
general protection fault, probably for non-canonical address 0x3de61901173d0bef: 0000 [#1] SMP NOPTI
CPU: 448 PID: 35281 Comm: setroubleshootd Kdump: loaded Tainted: G    B            --------- -  - 4.18.0-477.89.1.el8_8.x86_64 #1
   ...
RIP: 0010:kmem_cache_alloc+0xda/0x280
Code: 49 49 8b 50 08 49 8b 00 49 83 78 10 00 48 89 04 24 0f 84 6a 01 00 00 48 85 c0 0f 84 61 01 00 00 8b 4d 20 48 8b 7d 00 48 01 c1 <48> 8b 19 48 89 ce 48 33 9d 90 01 00 00 48 8d 4a 01 48 0f ce 48 31
RSP: 0018:ff6f9a3501a57d00 EFLAGS: 00010202
RAX: 3de61901173d0bcf RBX: 00000000006000c0 RCX: 3de61901173d0bef
RDX: 0000000000002140 RSI: 00000000006000c0 RDI: 0000000000039e40
RBP: ff4e5e89d000e800 R08: ff4e5e896f639e40 R09: 0000000000000000
R10: 0000000000000051 R11: 0000000000000000 R12: 00000000006000c0
R13: ffffffffb64f24bd R14: ff4e5e8a2edee3c0 R15: ff4e5e29ed489fb8
FS:  00007f7d65e9b700(0000) GS:ff4e5e896f600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7d6691a000 CR3: 00000078c3a78002 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
 anon_vma_fork+0x9d/0x120
 dup_mm+0x4b6/0x590
 copy_process+0x19dd/0x1cd0
 _do_fork+0x8b/0x340
 do_syscall_64+0x5b/0x1b0
 entry_SYSCALL_64_after_hwframe+0x61/0xc6
RIP: 0033:0x7f7d7ea7a913
Code: db 0f 85 28 01 00 00 64 4c 8b 0c 25 10 00 00 00 45 31 c0 4d 8d 91 d0 02 00 00 31 d2 31 f6 bf 11 00 20 01 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 b9 00 00 00 41 89 c5 85 c0 0f 85 c6 00 00
RSP: 002b:00007f7d65e983f0 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f7d7ea7a913
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
RBP: 0000000000000001 R08: 0000000000000000 R09: 00007f7d65e9b700
R10: 00007f7d65e9b9d0 R11: 0000000000000246 R12: 0000000000000004
R13: 00007f7d7fbf9b70 R14: 0000000000000003 R15: 00007f7d65eeb9d8
Modules linked in: ...

        ...
megaraid_sas 0000:61:00.0: scanning for scsi0...
megaraid_sas 0000:61:00.0: 14107 (790880616s/0x0001/CRIT) - VD ee/1 is now DEGRADED
mpt3sas_cm0: log_info(0x310f0400): originator(PL), code(0x0f), sub_code(0x0400)
mpt3sas_cm0: log_info(0x3003011d): originator(IOP), code(0x03), sub_code(0x011d)
mpt3sas_cm1: log_info(0x310f0400): originator(PL), code(0x0f), sub_code(0x0400)
mpt3sas_cm1: log_info(0x3003011d): originator(IOP), code(0x03), sub_code(0x011d)
mpt3sas_cm0: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
        ...
mpt3sas_cm1: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
mpt3sas_cm0: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
mpt3sas_cm0: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
mpt3sas_cm1: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
mpt3sas_cm1: log_info(0x30030109): originator(IOP), code(0x03), sub_code(0x0109)
iostat: Corrupted page table at address 5620ec48d6c0
PGD 628eefb067 P4D 61b27b0067 PUD 16973402677754b5 
Bad pagetable: 001d [#1] SMP NOPTI
CPU: 192 PID: 60873 Comm: iostat Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-477.10.1.el8_8.x86_
64 #1
   ...
RIP: 0033:0x5620ec48d6c0
iostat: Corrupted page table at address 5620ec48d696
PGD 628eefb067 P4D 61b27b0067 PUD 16973402677754b5 
Bad pagetable: 0009 [#2] SMP NOPTI
CPU: 192 PID: 60873 Comm: iostat Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-477.10.1.el8_8.x86_
64 #1
   ...
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 7
0 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad597d68 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad597da0
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad597c08 R12: 00fffffffffff000
R13: 00005620ec48d696 R14: ff5f3b78ad597da0 R15: 0000000000000000
FS:  00007f68b4e33540(0000) GS:ff1ae7a349600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005620ec48d696 CR3: 000000628ec52002 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 show_regs+0x2d/0x40
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 ? page_fault+0x8/0x30
 page_fault+0x1e/0x30
RIP: 0033:0x5620ec48d6c0
        ...
iostat: Corrupted page table at address 5620ec48d696
WARNING: stack recursion on stack type 5
BUG: stack guard page was hit at 0000000048852ff1 (stack is 00000000e335ca9e..00000000c65b8cbb)
kernel stack overflow (double-fault): 0000 [#18] SMP NOPTI
CPU: 192 PID: 60873 Comm: iostat Kdump: loaded Tainted: G           OE    --------- -  - 4.18.0-477.10.1.el8_8.x86_64 #1
   ...
RIP: 0010:cfb_imageblit+0x1c/0x4d0
Code: ff ff 49 89 d0 48 89 f5 e9 3f fb ff ff 90 0f 1f 44 00 00 8b 87 90 03 00 00 85 c0 0f 85 57 03 00 00 41 57 41 56 41 55 49 89 f5 <41> 54 49 89 fc 55 53 48 83 ec 38 8b 46 08 8b 76 04 44 8b 77 68 41
RSP: 0000:ff5f3b78ad594000 EFLAGS: 00010046
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000001
RDX: ff1ae7456f84a6c0 RSI: ff1ae7456f84a660 RDI: ff1ae744e3ab7400
RBP: 0000000000000001 R08: 0000000000000000 R09: 00000000ffffffff
R10: 0000000000000001 R11: 0000000000000000 R12: ff1ae7456f84a660
R13: ff1ae7456f84a660 R14: ff1ae744e3ab7400 R15: ff1ae7456f84a6b0
FS:  00007f68b4e33540(0000) GS:ff1ae7a349600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ff5f3b78ad593ff8 CR3: 000000628ec52002 CR4: 0000000000771ee0
PKRU: 55555554
Call Trace:
 soft_cursor+0x194/0x230
 bit_cursor+0x377/0x610
 ? get_color+0x26/0x120
 ? bit_putcs+0x550/0x550
 fbcon_scroll+0x9c/0xc20
 con_scroll+0x20f/0x230
 lf+0xa4/0xb0
 vt_console_print+0x314/0x400
 console_unlock+0x366/0x4b0
 vprintk_emit+0x151/0x250
 printk+0x58/0x73
 pgtable_bad+0x3d/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad594568 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad5945a0
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad594408 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad5945a0 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad5948e8 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad594920
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad594788 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad594920 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad594c68 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad594ca0
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad594b08 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad594ca0 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad594fe8 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad595020
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad594e88 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad595020 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad595368 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad5953a0
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad595208 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad5953a0 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad5956e8 EFLAGS: 00050046
RAX: 0000000000000002 RBX: ff1ae7471d764000 RCX: 0000000000000040
RDX: 0000000000000040 RSI: 00005620ec48d696 RDI: ff5f3b78ad595720
RBP: 0000000000000040 R08: 0000000000000000 R09: c0000000fffdffff
R10: 0000000000000001 R11: ff5f3b78ad595588 R12: ffffffffffffffff
R13: 00005620ec48d696 R14: ff5f3b78ad595720 R15: 0000000000000000
 __probe_kernel_read+0x54/0x90
 show_opcodes+0x56/0xa0
 ? irq_work_queue+0x9/0x30
 show_iret_regs+0x15/0x37
 __show_regs+0x1d/0x30
 ? page_fault+0x1e/0x30
 ? unwind_next_frame+0x334/0x540
 ? is_bpf_text_address+0xa/0x20
 show_trace_log_lvl+0x2c7/0x321
 ? page_fault+0x1e/0x30
 ? page_fault+0x1e/0x30
 __die_body+0x1a/0x60
 pgtable_bad+0x70/0x90
 __do_page_fault+0x2d8/0x450
 do_page_fault+0x37/0x130
 page_fault+0x1e/0x30
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40
Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f 01 ca e9 a3 7c 41 00 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 <f3> a4 31 c0 0f 01 ca e9 86 7c 41 00 66 0f 1f 44 00 00 83 f8 12 74
RSP: 0000:ff5f3b78ad595a68 EFLAGS: 00050046
RAX: 000000
Lost 5103 message(s)!