System crashed after many 'locks_remove_file+0x202' messages.

Solution Unverified - Updated -

Issue

  • System was crashed with the below error messages.
...
[677368.719162] BUG: unable to handle kernel NULL pointer dereference at 0000000000000060
[677368.728917] IP: [<ffffffffa8a69674>] d_path+0x34/0x150
[677368.735582] PGD af73c2f067 PUD a39077b067 PMD 0 
[677368.741658] Oops: 0000 [#1] SMP 
[677368.746177] Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag  ... ptp crct10dif_common megaraid_sas crc32c_intel nvme_core pps_core [last unloaded: tracedev]
[677368.867108] CPU: 39 PID: 12871 Comm: perl5.10-64 Kdump: loaded Tainted: P        W  OE  ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
[677368.882298] Hardware name: LENOVO System x3650 M5: -[8871AC1]-/01KN179, BIOS -[TCE148A-3.30]- 10/26/2020
[677368.893844] task: ffff9b9319a39080 ti: ffff9b9429148000 task.ti: ffff9b9429148000
[677368.903180] RIP: 0010:[<ffffffffa8a69674>]  [<ffffffffa8a69674>] d_path+0x34/0x150
[677368.912563] RSP: 0018:ffff9b942914be50  EFLAGS: 00010246
[677368.919411] RAX: ffff9ae3dee78000 RBX: ffff9b942914be90 RCX: 0000000000000000
[677368.928317] RDX: 0000000000001000 RSI: ffff9ae3dee77000 RDI: 0000000000000000
[677368.937179] RBP: ffff9b942914be80 R08: ffffffffa928edf7 R09: fffff0fdee7b9e00
[677368.946038] R10: 00000000000986e9 R11: ffff9b66bde1dd40 R12: ffff9ae3dee77000
[677368.954917] R13: ffff9b70fda24730 R14: 0000000000000fff R15: 00007ffd1594a340
[677368.963796] FS:  00007fdec47fd740(0000) GS:ffff9b94bf9c0000(0000) knlGS:0000000000000000
[677368.973753] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[677368.981172] CR2: 0000000000000060 CR3: 000000b14b8dc000 CR4: 00000000003607e0
[677368.990251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[677368.999155] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[677369.008264] Call Trace:
[677369.012081]  [<ffffffffa8ac9c55>] proc_pid_readlink+0xa5/0x110
[677369.019677]  [<ffffffffa8a549c1>] SyS_readlinkat+0x111/0x150
[677369.027107]  [<ffffffffa8a54a1b>] SyS_readlink+0x1b/0x20
[677369.034232]  [<ffffffffa8f95f92>] system_call_fastpath+0x25/0x2a
[677369.042032] Code: 48 01 f0 48 89 e5 53 48 89 fb 48 83 ec 28 65 48 8b 3c 25 28 00 00 00 48 89 7c 24 20 31 ff 48 8b 7b 08 89 54 24 04 48 89 44 24 08 <48> 8b 4f 60 48 85 c9 74 43 48 8b 49 40 48 85 c9 74 3a 48 3b 7f 
[677369.065890] RIP  [<ffffffffa8a69674>] d_path+0x34/0x150
[677369.072841]  RSP <ffff9b942914be50>
[677369.077780] CR2: 0000000000000060
  • Before the crash, there were many below type of warning messages.
...
[677277.770211] ------------[ cut here ]------------
[677277.776815] WARNING: CPU: 0 PID: 28073 at fs/locks.c:2443 locks_remove_file+0x202/0x210
[677277.787973] leftover lock: dev=0:42 ino=34005122 type=1 flags=0x1 start=0 end=9223372036854775807
[677277.800139] Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag ... tg3 devlink nvme crct10dif_pclmul ptp crct10dif_common megaraid_sas crc32c_intel nvme_core pps_core [last unloaded: tracedev]
[677277.932244] CPU: 0 PID: 28073 Comm: gpfs.ganesha.nf Kdump: loaded Tainted: P        W  OE  ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
[677277.950651] Hardware name: LENOVO System x3650 M5: -[8871AC1]-/01KN179, BIOS -[TCE148A-3.30]- 10/26/2020
[677277.963759] Call Trace:
[677277.968904]  [<ffffffffa8f83579>] dump_stack+0x19/0x1b
[677277.977072]  [<ffffffffa889b278>] __warn+0xd8/0x100
[677277.984941]  [<ffffffffa889b2ff>] warn_slowpath_fmt+0x5f/0x80
[677277.993793]  [<ffffffffa8aaf122>] locks_remove_file+0x202/0x210
[677278.002847]  [<ffffffffc0cd6887>] ? ss_fs_unlocked_ioctl+0x107/0x580 [mmfslinux]
[677278.013661]  [<ffffffffa8a5085d>] __fput+0xbd/0x230
[677278.021639]  [<ffffffffa8a50abe>] ____fput+0xe/0x10
[677278.029473]  [<ffffffffa88c299b>] task_work_run+0xbb/0xe0
[677278.037847]  [<ffffffffa882cc65>] do_notify_resume+0xa5/0xc0
[677278.046463]  [<ffffffffa8f962ef>] int_signal+0x12/0x17
[677278.054449] ---[ end trace 4f271e130c071185 ]---
...

Environment

  • Red Hat Enterprise Linux 7
  • IBM gpfs filesystem is in use

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content