RHEL9: Kernel panic at __list_add_valid() due to NULL pointer with xprt_complete_bc_request()

Issue

• Kernel panics with the following messages:

[4791196.224287] BUG: kernel NULL pointer dereference, address: 0000000000000008
[4791196.224290] #PF: supervisor read access in kernel mode
[4791196.224292] #PF: error_code(0x0000) - not-present page
[4791196.224294] PGD 178d16067 P4D 178d16067 PUD 1588da067 PMD 0 
[4791196.224298] Oops: 0000 [#1] PREEMPT SMP NOPTI
[4791196.224300] CPU: 2 PID: 2094163 Comm: kworker/u48:5 Kdump: loaded Not tainted 5.14.0-503.15.1.el9_5.x86_64 #1
[4791196.224303] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
[4791196.224304] Workqueue: xprtiod xs_stream_data_receive_workfn [sunrpc]
[4791196.224381] RIP: 0010:__list_add_valid+0x3/0x40
[4791196.224386] Code: ff ff ff e9 a5 fe ff ff 4d 89 ea eb bb 48 89 44 24 40 4d 89 ea eb b1 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 49 89 d0 <48> 8b 52 08 48 39 f2 0f 85 8e bd 64 00 4c 8b 0a 4d 39 c1 0f 85 aa
[4791196.224388] RSP: 0018:ffffbf0b4777fd40 EFLAGS: 00010246
[4791196.224389] RAX: 0000000000000000 RBX: ffff9dc513e3be00 RCX: 0000000000000002
[4791196.224390] RDX: 0000000000000000 RSI: ffff9db7b8e428e8 RDI: ffff9dc513e3bfb8
[4791196.224391] RBP: ffff9dc513e3bfb8 R08: 0000000000000000 R09: 0000000000000000
[4791196.224391] R10: 000036666d367701 R11: 0000000000000000 R12: ffff9db7b8e42840
[4791196.224392] R13: 0000000000000000 R14: ffff9db7b8e428f8 R15: ffff9db7b8e428e8
[4791196.224393] FS:  0000000000000000(0000) GS:ffff9dc73fa80000(0000) knlGS:0000000000000000
[4791196.224394] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[4791196.224395] CR2: 0000000000000008 CR3: 000000010b16a001 CR4: 00000000007706f0
[4791196.224422] PKRU: 55555554
[4791196.224423] Call Trace:
[4791196.224426]  <TASK>
[4791196.224430]  ? show_trace_log_lvl+0x1c4/0x2df
[4791196.224437]  ? show_trace_log_lvl+0x1c4/0x2df
[4791196.224440]  ? xprt_complete_bc_request+0xd7/0x120 [sunrpc]
[4791196.224491]  ? __die_body.cold+0x8/0xd
[4791196.224493]  ? page_fault_oops+0x134/0x170
[4791196.224497]  ? sock_recvmsg+0x95/0xa0
[4791196.224502]  ? exc_page_fault+0x62/0x150
[4791196.224507]  ? asm_exc_page_fault+0x22/0x30
[4791196.224513]  ? __list_add_valid+0x3/0x40
[4791196.224514]  xprt_complete_bc_request+0xd7/0x120 [sunrpc]
[4791196.224559]  xs_read_stream.constprop.0+0x3b2/0x470 [sunrpc]
[4791196.224591]  ? __switch_to_asm+0x3a/0x80
[4791196.224595]  ? finish_task_switch.isra.0+0x8c/0x2a0
[4791196.224600]  xs_stream_data_receive_workfn+0x63/0x130 [sunrpc]
[4791196.224633]  process_one_work+0x194/0x380
[4791196.224637]  worker_thread+0x2fe/0x410
[4791196.224639]  ? __pfx_worker_thread+0x10/0x10
[4791196.224642]  kthread+0xdd/0x100
[4791196.224646]  ? __pfx_kthread+0x10/0x10
[4791196.224649]  ret_from_fork+0x29/0x50
[4791196.224651]  </TASK>
[4791196.224651] Modules linked in: binfmt_misc tls ext4 mbcache jbd2 rpcsec_gss_krb5 auth_rpcgss nfsv4 nfs lockd grace fscache netfs nls_utf8 cifs cifs_arc4 rdma_cm iw_cm ib_cm ib_core cifs_md4 dns_resolver xt_TPROXY nf_tproxy_ipv6 nf_tproxy_ipv4 ip_set nft_chain_nat xt_nat xt_MASQUERADE xt_CT xt_mark cls_bpf sch_ingress vxlan ip6_udp_tunnel udp_tunnel veth xt_socket nf_socket_ipv4 nf_socket_ipv6 ip6table_filter ip6table_raw ip6table_mangle ip6_tables iptable_filter iptable_raw iptable_mangle iptable_nat nf_nat ip_tables xt_conntrack xt_comment nft_compat nft_counter nf_tables nfnetlink iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi overlay rfkill vsock_loopback vmw_vsock_virtio_transport_common vmw_vsock_vmci_transport vsock sunrpc dm_multipath intel_rapl_msr intel_rapl_common intel_uncore_frequency_common isst_if_mbox_msr isst_if_common nfit vmwgfx libnvdimm vmw_balloon rapl drm_ttm_helper ttm drm_kms_helper pcspkr vmw_vmci i2c_piix4 joydev drm xfs sr_mod cdrom ata_generic sd_mod ata_piix t10_pi crct10dif_pclmul
[4791196.224701]  sg crc32_pclmul libata vmxnet3 ghash_clmulni_intel vmw_pvscsi serio_raw dm_mirror dm_region_hash dm_log dm_mod fuse br_netfilter bridge stp llc ip_vs_wrr ip_vs_wlc ip_vs_sh ip_vs_rr ip_vs_lc ip_vs nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc32c_intel
[4791196.224729] CR2: 0000000000000008