RHEL 8.4: Kernel crashes at xdr_set_page_base due to NULL pointer dereference

Solution Verified - Updated -

Issue

  • Kernel crashes with the following log messages.

    [  477.872990] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
    [  477.873022] PGD 0 P4D 0 
    [  477.873032] Oops: 0000 [#1] SMP PTI
    [  477.873045] CPU: 3 PID: 59 Comm: kworker/u16:2 Kdump: loaded Not tainted 4.18.0-305.el8.x86_64 #1
    [  477.873076] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
    [  477.873141] Workqueue: rpciod rpc_async_schedule [sunrpc]
    [  477.873167] RIP: 0010:xdr_set_page_base+0x3d/0x80 [sunrpc]
    [  477.873184] Code: 29 f1 89 d0 39 d1 0f 46 c1 89 f1 41 03 48 30 49 8b 70 28 89 ca 81 e1 ff 0f 00 00 c1 ea 0c 48 8d 14 d6 89 ce 01 c1 48 89 57 30 <48> 8b 12 48 2b 15 c1 55 db e4 48 c7 47 18 00 00 00 00 48 c1 fa 06
    [  477.873229] RSP: 0018:ffffba5ac3367c78 EFLAGS: 00010246
    [  477.873244] RAX: 0000000000000000 RBX: ffffba5ac3367db8 RCX: 0000000000000000
    [  477.873263] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffba5ac3367db8
    [  477.873282] RBP: 0000000000000000 R08: ffff94716da4d250 R09: 0000000000000000
    [  477.873300] R10: 8080808080808080 R11: ffff9471a7ce8be4 R12: 0000000000000009
    [  477.873318] R13: ffffba5ac3367db8 R14: ffff94717468d800 R15: ffffffffc07a2f50
    [  477.873338] FS:  0000000000000000(0000) GS:ffff9471a7cc0000(0000)     knlGS:0000000000000000
    [  477.873358] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [  477.873374] CR2: 0000000000000000 CR3: 00000000b2610003 CR4: 00000000001706e0
    [  477.873426] Call Trace:
    [  477.873450]  xdr_set_next_buffer+0xab/0x100 [sunrpc]
    [  477.873474]  xdr_inline_decode+0x169/0x1d0 [sunrpc]
    [  477.873511]  __decode_op_hdr+0x22/0x120 [nfsv4]
    [  477.873531]  ? rpc_decode_header+0x560/0x560 [sunrpc]
    [  477.873552]  decode_getfattr_generic.constprop.124+0x52/0x240 [nfsv4]
    [  477.873578]  ? __rpc_sleep_on_priority_timeout+0xe0/0xe0 [sunrpc]
    [  477.873602]  ? rpc_decode_header+0x560/0x560 [sunrpc]
    [  477.873622]  nfs4_xdr_dec_open_noattr+0xc7/0x100 [nfsv4]
    [  477.873643]  ? rpc_decode_header+0xec/0x560 [sunrpc]
    [  477.873663]  call_decode+0x1f4/0x220 [sunrpc]
    [  477.873684]  __rpc_execute+0x85/0x3c0 [sunrpc]
    [  477.873704]  rpc_async_schedule+0x29/0x40 [sunrpc]
    [  477.873721]  process_one_work+0x1a7/0x360
    [  477.873734]  worker_thread+0x30/0x390
    [  477.873746]  ? create_worker+0x1a0/0x1a0
    [  477.873758]  kthread+0x116/0x130
    [  477.873770]  ? kthread_flush_work_fn+0x10/0x10
    [  477.873786]  ret_from_fork+0x35/0x40
    [  477.873797] Modules linked in: rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache nf_nat_ftp nft_objref nf_conntrack_ftp nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nf_tables_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink sunrpc intel_rapl_msr intel_rapl_common sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel rapl vmw_balloon vmw_vmci pcspkr joydev i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod t10_pi sg ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crc32c_intel vmw_pvscsi serio_raw vmxnet3 dm_mirror dm_region_hash dm_log dm_mod fuse
    [  477.873987] CR2: 0000000000000000
    
  • Another pattern of crash with nfs3:

[364802.855064] nfs: server hkgs00500703 OK
[364802.858424] nfs: server hkgs00500703 OK
[382167.227312] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
[382167.235237] PGD 0 P4D 0 
[382167.237870] Oops: 0000 [#1] SMP NOPTI
[382167.241622] CPU: 8 PID: 128947 Comm: grep Kdump: loaded Not tainted 4.18.0-305.el8.x86_64 #1
[382167.250147] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS 2.11.2 004/21/2021
[382167.257910] RIP: 0010:xdr_set_page_base+0x3d/0x80 [sunrpc]
[382167.263492] Code: 29 f1 89 d0 39 d1 0f 46 c1 89 f1 41 03 48 30 49 8b 70 28 89 ca 81 e1 ff 0f 00 00 c1 ea 0c 48 8d 14 d6 89 ce 01 c1 48 89 57 30 <48> 8b 12 48 2b 15 c1 b5 0a dc 48 c7 47 18 00 00 00 00 48 c1 fa 06
[382167.282354] RSP: 0018:ffffbabcce92fb58 EFLAGS: 00010246
[382167.287671] RAX: 0000000000000000 RBX: ffffbabcce92fbf8 RCX: 0000000000000000
[382167.294904] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffbabcce92fbf8
[382167.302132] RBP: 0000000000000000 R08: ffff902ee8765c50 R09: 0000000000000000
[382167.309357] R10: 0000000000000000 R11: ffff902b0c128be4 R12: ffffffff9d051540
[382167.316587] R13: ffff902ee8765c00 R14: ffffffffc0ab9b20 R15: ffffffffc0aacf50
[382167.323816] FS:  00007fae5266c2c0(0000) GS:ffff902b0c100000(0000) knlGS:0000000000000000
[382167.331999] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[382167.337835] CR2: 0000000000000000 CR3: 0000000120be8005 CR4: 00000000007706e0
[382167.345066] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[382167.352293] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[382167.359521] PKRU: 55555554
[382167.362323] Call Trace:
[382167.364880]  xdr_set_next_buffer+0xab/0x100 [sunrpc]
[382167.369945]  xdr_inline_decode+0x169/0x1d0 [sunrpc]
[382167.374922]  decode_post_op_attr+0x1c/0x50 [nfsv3]
[382167.379808]  nfs3_xdr_dec_fsstat3res+0x7d/0x110 [nfsv3]
[382167.385135]  call_decode+0x1f4/0x220 [sunrpc]
[382167.389589]  __rpc_execute+0x85/0x3c0 [sunrpc]
[382167.394130]  rpc_execute+0xb6/0xd0 [sunrpc]
[382167.398411]  rpc_run_task+0x144/0x190 [sunrpc]
[382167.402950]  rpc_call_sync+0x50/0x90 [sunrpc]
[382167.407402]  nfs3_rpc_wrapper+0x20/0xa0 [nfsv3]
[382167.412031]  nfs3_proc_statfs+0x5e/0x90 [nfsv3]
[382167.416667]  nfs_statfs+0x6c/0x170 [nfs]
[382167.420689]  statfs_by_dentry+0x67/0x90
[382167.424623]  vfs_statfs+0x16/0xc0
[382167.428035]  fd_statfs+0x2d/0x60
[382167.431358]  ? iterate_dir+0xe0/0x190
[382167.435118]  __do_sys_fstatfs+0x20/0x50
[382167.439052]  do_syscall_64+0x5b/0x1a0
[382167.442815]  entry_SYSCALL_64_after_hwframe+0x65/0xca
[382167.447964] RIP: 0033:0x7fae51f0700b
[382167.451636] Code: 73 01 c3 48 8b 0d 7d 1e 2d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 8a 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4d 1e 2d 00 f7 d8 64 89 01 48
[382167.470887] RSP: 002b:00007fff100fdbe8 EFLAGS: 00000202 ORIG_RAX: 000000000000008a
[382167.478920] RAX: ffffffffffffffda RBX: 000056480ac299b0 RCX: 00007fae51f0700b
[382167.486513] RDX: 000056480ac27840 RSI: 00007fff100fdc00 RDI: 0000000000000004
[382167.494102] RBP: 000056480ac294d0 R08: 000056480ac33e50 R09: 0000000000000003
[382167.501682] R10: 0000000000000001 R11: 0000000000000202 R12: 000056480ac28210
[382167.509258] R13: 000056480ac27840 R14: 000056480ac27780 R15: 000056480ac28210
[382167.516822] Modules linked in: fuse binfmt_misc md4 nfsv3 nfs_acl sha512_ssse3 sha512_generic cmac nls_utf8 cifs rpcsec_gss_krb5 auth_rpcgss rdma_cm iw_cm ib_cm nfsv4 ib_core libarc4 dns_resolver nfs lockd grace fscache mpt3sas raid_class scsi_transport_sas dell_rbu sunrpc intel_rapl_msr intel_rapl_common isst_if_common skx_edac nfit libnvdimm ipmi_ssif x86_pkg_temp_thermal coretemp kvm_intel iTCO_wdt wmi_bmof kvm iTCO_vendor_support dell_smbios dell_wmi_descriptor irqbypass dcdbas crct10dif_pclmul crc32_pclmul sr_mod ghash_clmulni_intel rapl cdrom intel_cstate pcspkr intel_uncore mei_me lpc_ich acpi_ipmi mei i2c_i801 wmi ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter ip_tables uas usb_storage xfs libcrc32c sd_mod t10_pi sg mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm igb megaraid_sas dca crc32c_intel i2c_algo_bit dm_mirror dm_region_hash dm_log dm_mod
[382167.598353] CR2: 0000000000000000

Environment

  • Red Hat Enterprise Linux 8.4 (NFS client)
  • kernel >= 4.18.0-305.el8 and kernel < 4.18.0-305.17.1.el8_4
  • NFSv4 / Seen on NFSv3

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content