RHEL8: Kernel panic due to nlm_lockowner.host being corrupted
Issue
- Kernel panic with logs:
[8125134.026645] LustreError: 3951555:0:(llite_nfs.c:326:ll_dir_get_parent_fid()) Skipped 335 previous similar messages
[8130123.237154] BUG: unable to handle kernel paging request at 0000000065cd6512
[8130123.244687] PGD 0 P4D 0
[8130123.247734] Oops: 0000 [#1] SMP NOPTI
[8130123.251820] CPU: 6 PID: 3951297 Comm: lockd Kdump: loaded Tainted: G W IOE --------- - - 4.18.0-477.10.1.el8_8.x86_64 #1
[8130123.264306] Hardware name: /0H28RR, BIOS 2.13.3 12/13/2021
[8130123.270296] RIP: 0010:nlmsvc_mark_host+0x11/0x30 [lockd]
[8130123.276032] Code: fe ff ff 66 0f 1f 44 00 00 0f 1f 44 00 00 b8 01 00 00 00 e9 b1 3d 2d f6 90 0f 1f 44 00 00 48 8b 86 00 02 00 00 48 85 c0 74 10 <48> 3b 87 00 02 00 00 74 07 31 c0 e9 8f 3d 2d f6 80 8f 36 01 00 00
[8130123.295482] RSP: 0018:ffffac7d90f778d0 EFLAGS: 00010286
[8130123.301136] RAX: ffffffffb8932700 RBX: ffff8e07b86dca20 RCX: ffffffffc153f280
[8130123.308699] RDX: 0000000000000001 RSI: ffffac7d90f77a30 RDI: 0000000065cd6312
[8130123.316264] RBP: ffff8e1346842cc8 R08: ffff8e16b94200c0 R09: ffff8e1904a38600
[8130123.323827] R10: 0000000000000000 R11: fefefefefefefeff R12: ffff8e1346842cb0
[8130123.331382] R13: ffffac7d90f77a30 R14: ffffffffc152d690 R15: ffff8e11afd76c00
[8130123.338941] FS: 0000000000000000(0000) GS:ffff8e363f580000(0000) knlGS:0000000000000000
[8130123.347454] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[8130123.353614] CR2: 0000000065cd6512 CR3: 00000016a5210006 CR4: 00000000007706e0
[8130123.361168] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[8130123.368725] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[8130123.376278] PKRU: 55555554
[8130123.379394] Call Trace:
[8130123.382251] nlm_traverse_locks+0xba/0x170 [lockd]
[8130123.387452] ? netif_rx_internal+0x41/0x100
[8130123.392043] ? loopback_xmit+0xd8/0x130
[8130123.396287] ? _cond_resched+0x15/0x30
[8130123.400435] ? mutex_lock+0xe/0x30
[8130123.404236] ? nlmsvc_traverse_blocks+0x37/0xc0 [lockd]
[8130123.409865] ? nlmsvc_always_match+0x10/0x10 [lockd]
[8130123.415225] nlm_traverse_files+0x99/0x190 [lockd]
[8130123.420418] nlmsvc_mark_resources+0x51/0x80 [lockd]
[8130123.425779] ? arch_stack_walk+0x66/0x100
[8130123.430176] ? orc_find.part.4+0xfb/0x1a0
[8130123.434569] ? ret_from_fork+0x1f/0x40
[8130123.438698] ? unwind_next_frame+0x5f/0x540
[8130123.443263] ? profile_setup.cold.9+0xa1/0xa1
[8130123.448004] ? arch_stack_walk+0x8f/0x100
[8130123.452387] ? deactivate_slab+0x58d/0x640
[8130123.456856] ? _copy_to_iter+0x14f/0x3d0
[8130123.461151] ? sock_alloc_inode+0x36/0xb0
[8130123.465533] ? stack_trace_save+0x4b/0x70
[8130123.469907] ? __check_object_size+0xac/0x173
[8130123.474630] ? __skb_datagram_iter+0x1a6/0x310
[8130123.479439] ? _copy_to_iter+0x304/0x3d0
[8130123.483724] ? __check_object_size+0xac/0x173
[8130123.488439] ? free_unref_page_commit+0x91/0x110
[8130123.493410] ? tcp_recvmsg+0x7a9/0xa90
[8130123.497514] ? kmem_cache_free+0x2d6/0x300
[8130123.501960] nlm_gc_hosts+0x76/0x130 [lockd]
[8130123.506576] nlmsvc_lookup_host+0xe8/0x3a0 [lockd]
[8130123.511719] ? sunrpc_cache_lookup_rcu+0x73/0x320 [sunrpc]
[8130123.517584] nlm4svc_retrieve_args+0x50/0x150 [lockd]
[8130123.522979] __nlm4svc_proc_lock+0x71/0x120 [lockd]
[8130123.528202] nlmsvc_dispatch+0xa2/0x1b0 [lockd]
[8130123.533074] svc_process_common+0x360/0x5c0 [sunrpc]
[8130123.538400] ? svc_sock_secure_port+0x12/0x40 [sunrpc]
[8130123.543888] ? nlm_shutdown_hosts+0x20/0x20 [lockd]
[8130123.549095] ? grace_ender+0x20/0x20 [lockd]
[8130123.553692] svc_process+0xb7/0xf0 [sunrpc]
[8130123.558224] lockd+0xae/0x190 [lockd]
[8130123.562218] ? __kthread_parkme+0x4b/0x70
[8130123.566550] kthread+0x134/0x150
[8130123.570100] ? set_kthread_struct+0x50/0x50
[8130123.574600] ret_from_fork+0x1f/0x40
[8130123.578497] Modules linked in: mgc(OE) lustre(OE) lmv(OE) mdc(OE) fid(OE) osc(OE) lov(OE) fld(OE) ptlrpc(OE) obdclass(OE) ko2iblnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nft_counter iscsi_target_mod target_core_mod nf_tables libcrc32c nfnetlink dm_mod binfmt_misc overlay tcp_diag inet_diag nfsd auth_rpcgss nfs_acl lockd grace bonding mpt3sas raid_class scsi_transport_sas uio_pci_generic uio vfio_pci vfio_virqfd vfio_iommu_type1 vfio cuse dell_rbu rdma_ucm(OE) rdma_cm(OE) iw_cm(OE) dcdbas ib_ipoib(OE) ib_cm(OE) ib_umad(OE) sunrpc intel_rapl_msr intel_rapl_common isst_if_common skx_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel rapl dell_smbios iTCO_wdt intel_cstate iTCO_vendor_support intel_uncore wmi_bmof dell_wmi_descriptor pcspkr acpi_ipmi mei_me ipmi_si mei lpc_ich ipmi_devintf i2c_i801 ipmi_msghandler wmi acpi_power_meter vfat fat sch_fq tcp_htcp
[8130123.578560] knem(OE) ext4 mbcache jbd2 sd_mod t10_pi sg mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) mgag200 i2c_algo_bit drm_shmem_helper drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm ahci mlx5_core(OE) bnxt_en libahci mlxfw(OE) pci_hyperv_intf psample megaraid_sas libata mlxdevm(OE) mlx_compat(OE) crc32c_intel tls tg3 xpmem(OE) fuse [last unloaded: libcfs]
[8130123.700630] Red Hat flags: eBPF/rawtrace
[8130123.704904] CR2: 0000000065cd6512
Environment
- Red Hat Enterprise Linux 8.8
- [lustre]
2.12.9_ddn45
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.