RHEL8: Kernel panic due to nlm_lockowner.host being corrupted

Solution Verified - Updated -

Issue

  • Kernel panic with logs:
[8125134.026645] LustreError: 3951555:0:(llite_nfs.c:326:ll_dir_get_parent_fid()) Skipped 335 previous similar messages
[8130123.237154] BUG: unable to handle kernel paging request at 0000000065cd6512
[8130123.244687] PGD 0 P4D 0 
[8130123.247734] Oops: 0000 [#1] SMP NOPTI
[8130123.251820] CPU: 6 PID: 3951297 Comm: lockd Kdump: loaded Tainted: G        W IOE    --------- -  - 4.18.0-477.10.1.el8_8.x86_64 #1
[8130123.264306] Hardware name:  /0H28RR, BIOS 2.13.3 12/13/2021
[8130123.270296] RIP: 0010:nlmsvc_mark_host+0x11/0x30 [lockd]
[8130123.276032] Code: fe ff ff 66 0f 1f 44 00 00 0f 1f 44 00 00 b8 01 00 00 00 e9 b1 3d 2d f6 90 0f 1f 44 00 00 48 8b 86 00 02 00 00 48 85 c0 74 10 <48> 3b 87 00 02 00 00 74 07 31 c0 e9 8f 3d 2d f6 80 8f 36 01 00 00
[8130123.295482] RSP: 0018:ffffac7d90f778d0 EFLAGS: 00010286
[8130123.301136] RAX: ffffffffb8932700 RBX: ffff8e07b86dca20 RCX: ffffffffc153f280
[8130123.308699] RDX: 0000000000000001 RSI: ffffac7d90f77a30 RDI: 0000000065cd6312
[8130123.316264] RBP: ffff8e1346842cc8 R08: ffff8e16b94200c0 R09: ffff8e1904a38600
[8130123.323827] R10: 0000000000000000 R11: fefefefefefefeff R12: ffff8e1346842cb0
[8130123.331382] R13: ffffac7d90f77a30 R14: ffffffffc152d690 R15: ffff8e11afd76c00
[8130123.338941] FS:  0000000000000000(0000) GS:ffff8e363f580000(0000) knlGS:0000000000000000
[8130123.347454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[8130123.353614] CR2: 0000000065cd6512 CR3: 00000016a5210006 CR4: 00000000007706e0
[8130123.361168] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[8130123.368725] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[8130123.376278] PKRU: 55555554
[8130123.379394] Call Trace:
[8130123.382251]  nlm_traverse_locks+0xba/0x170 [lockd]
[8130123.387452]  ? netif_rx_internal+0x41/0x100
[8130123.392043]  ? loopback_xmit+0xd8/0x130
[8130123.396287]  ? _cond_resched+0x15/0x30
[8130123.400435]  ? mutex_lock+0xe/0x30
[8130123.404236]  ? nlmsvc_traverse_blocks+0x37/0xc0 [lockd]
[8130123.409865]  ? nlmsvc_always_match+0x10/0x10 [lockd]
[8130123.415225]  nlm_traverse_files+0x99/0x190 [lockd]
[8130123.420418]  nlmsvc_mark_resources+0x51/0x80 [lockd]
[8130123.425779]  ? arch_stack_walk+0x66/0x100
[8130123.430176]  ? orc_find.part.4+0xfb/0x1a0
[8130123.434569]  ? ret_from_fork+0x1f/0x40
[8130123.438698]  ? unwind_next_frame+0x5f/0x540
[8130123.443263]  ? profile_setup.cold.9+0xa1/0xa1
[8130123.448004]  ? arch_stack_walk+0x8f/0x100
[8130123.452387]  ? deactivate_slab+0x58d/0x640
[8130123.456856]  ? _copy_to_iter+0x14f/0x3d0
[8130123.461151]  ? sock_alloc_inode+0x36/0xb0
[8130123.465533]  ? stack_trace_save+0x4b/0x70
[8130123.469907]  ? __check_object_size+0xac/0x173
[8130123.474630]  ? __skb_datagram_iter+0x1a6/0x310
[8130123.479439]  ? _copy_to_iter+0x304/0x3d0
[8130123.483724]  ? __check_object_size+0xac/0x173
[8130123.488439]  ? free_unref_page_commit+0x91/0x110
[8130123.493410]  ? tcp_recvmsg+0x7a9/0xa90
[8130123.497514]  ? kmem_cache_free+0x2d6/0x300
[8130123.501960]  nlm_gc_hosts+0x76/0x130 [lockd]
[8130123.506576]  nlmsvc_lookup_host+0xe8/0x3a0 [lockd]
[8130123.511719]  ? sunrpc_cache_lookup_rcu+0x73/0x320 [sunrpc]
[8130123.517584]  nlm4svc_retrieve_args+0x50/0x150 [lockd]
[8130123.522979]  __nlm4svc_proc_lock+0x71/0x120 [lockd]
[8130123.528202]  nlmsvc_dispatch+0xa2/0x1b0 [lockd]
[8130123.533074]  svc_process_common+0x360/0x5c0 [sunrpc]
[8130123.538400]  ? svc_sock_secure_port+0x12/0x40 [sunrpc]
[8130123.543888]  ? nlm_shutdown_hosts+0x20/0x20 [lockd]
[8130123.549095]  ? grace_ender+0x20/0x20 [lockd]
[8130123.553692]  svc_process+0xb7/0xf0 [sunrpc]
[8130123.558224]  lockd+0xae/0x190 [lockd]
[8130123.562218]  ? __kthread_parkme+0x4b/0x70
[8130123.566550]  kthread+0x134/0x150
[8130123.570100]  ? set_kthread_struct+0x50/0x50
[8130123.574600]  ret_from_fork+0x1f/0x40
[8130123.578497] Modules linked in: mgc(OE) lustre(OE) lmv(OE) mdc(OE) fid(OE) osc(OE) lov(OE) fld(OE) ptlrpc(OE) obdclass(OE) ko2iblnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nft_counter iscsi_target_mod target_core_mod nf_tables libcrc32c nfnetlink dm_mod binfmt_misc overlay tcp_diag inet_diag nfsd auth_rpcgss nfs_acl lockd grace bonding mpt3sas raid_class scsi_transport_sas uio_pci_generic uio vfio_pci vfio_virqfd vfio_iommu_type1 vfio cuse dell_rbu rdma_ucm(OE) rdma_cm(OE) iw_cm(OE) dcdbas ib_ipoib(OE) ib_cm(OE) ib_umad(OE) sunrpc intel_rapl_msr intel_rapl_common isst_if_common skx_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel rapl dell_smbios iTCO_wdt intel_cstate iTCO_vendor_support intel_uncore wmi_bmof dell_wmi_descriptor pcspkr acpi_ipmi mei_me ipmi_si mei lpc_ich ipmi_devintf i2c_i801 ipmi_msghandler wmi acpi_power_meter vfat fat sch_fq tcp_htcp
[8130123.578560]  knem(OE) ext4 mbcache jbd2 sd_mod t10_pi sg mlx5_ib(OE) ib_uverbs(OE) ib_core(OE) mgag200 i2c_algo_bit drm_shmem_helper drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm ahci mlx5_core(OE) bnxt_en libahci mlxfw(OE) pci_hyperv_intf psample megaraid_sas libata mlxdevm(OE) mlx_compat(OE) crc32c_intel tls tg3 xpmem(OE) fuse [last unloaded: libcfs]
[8130123.700630] Red Hat flags: eBPF/rawtrace
[8130123.704904] CR2: 0000000065cd6512

Environment

  • Red Hat Enterprise Linux 8.8
  • [lustre] 2.12.9_ddn45

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content