System crashed after many 'locks_remove_file+0x202' messages.
Issue
- System was crashed with the below error messages.
...
[677368.719162] BUG: unable to handle kernel NULL pointer dereference at 0000000000000060
[677368.728917] IP: [<ffffffffa8a69674>] d_path+0x34/0x150
[677368.735582] PGD af73c2f067 PUD a39077b067 PMD 0
[677368.741658] Oops: 0000 [#1] SMP
[677368.746177] Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag ... ptp crct10dif_common megaraid_sas crc32c_intel nvme_core pps_core [last unloaded: tracedev]
[677368.867108] CPU: 39 PID: 12871 Comm: perl5.10-64 Kdump: loaded Tainted: P W OE ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
[677368.882298] Hardware name: LENOVO System x3650 M5: -[8871AC1]-/01KN179, BIOS -[TCE148A-3.30]- 10/26/2020
[677368.893844] task: ffff9b9319a39080 ti: ffff9b9429148000 task.ti: ffff9b9429148000
[677368.903180] RIP: 0010:[<ffffffffa8a69674>] [<ffffffffa8a69674>] d_path+0x34/0x150
[677368.912563] RSP: 0018:ffff9b942914be50 EFLAGS: 00010246
[677368.919411] RAX: ffff9ae3dee78000 RBX: ffff9b942914be90 RCX: 0000000000000000
[677368.928317] RDX: 0000000000001000 RSI: ffff9ae3dee77000 RDI: 0000000000000000
[677368.937179] RBP: ffff9b942914be80 R08: ffffffffa928edf7 R09: fffff0fdee7b9e00
[677368.946038] R10: 00000000000986e9 R11: ffff9b66bde1dd40 R12: ffff9ae3dee77000
[677368.954917] R13: ffff9b70fda24730 R14: 0000000000000fff R15: 00007ffd1594a340
[677368.963796] FS: 00007fdec47fd740(0000) GS:ffff9b94bf9c0000(0000) knlGS:0000000000000000
[677368.973753] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[677368.981172] CR2: 0000000000000060 CR3: 000000b14b8dc000 CR4: 00000000003607e0
[677368.990251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[677368.999155] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[677369.008264] Call Trace:
[677369.012081] [<ffffffffa8ac9c55>] proc_pid_readlink+0xa5/0x110
[677369.019677] [<ffffffffa8a549c1>] SyS_readlinkat+0x111/0x150
[677369.027107] [<ffffffffa8a54a1b>] SyS_readlink+0x1b/0x20
[677369.034232] [<ffffffffa8f95f92>] system_call_fastpath+0x25/0x2a
[677369.042032] Code: 48 01 f0 48 89 e5 53 48 89 fb 48 83 ec 28 65 48 8b 3c 25 28 00 00 00 48 89 7c 24 20 31 ff 48 8b 7b 08 89 54 24 04 48 89 44 24 08 <48> 8b 4f 60 48 85 c9 74 43 48 8b 49 40 48 85 c9 74 3a 48 3b 7f
[677369.065890] RIP [<ffffffffa8a69674>] d_path+0x34/0x150
[677369.072841] RSP <ffff9b942914be50>
[677369.077780] CR2: 0000000000000060
- Before the crash, there were many below type of warning messages.
...
[677277.770211] ------------[ cut here ]------------
[677277.776815] WARNING: CPU: 0 PID: 28073 at fs/locks.c:2443 locks_remove_file+0x202/0x210
[677277.787973] leftover lock: dev=0:42 ino=34005122 type=1 flags=0x1 start=0 end=9223372036854775807
[677277.800139] Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag ... tg3 devlink nvme crct10dif_pclmul ptp crct10dif_common megaraid_sas crc32c_intel nvme_core pps_core [last unloaded: tracedev]
[677277.932244] CPU: 0 PID: 28073 Comm: gpfs.ganesha.nf Kdump: loaded Tainted: P W OE ------------ T 3.10.0-1160.53.1.el7.x86_64 #1
[677277.950651] Hardware name: LENOVO System x3650 M5: -[8871AC1]-/01KN179, BIOS -[TCE148A-3.30]- 10/26/2020
[677277.963759] Call Trace:
[677277.968904] [<ffffffffa8f83579>] dump_stack+0x19/0x1b
[677277.977072] [<ffffffffa889b278>] __warn+0xd8/0x100
[677277.984941] [<ffffffffa889b2ff>] warn_slowpath_fmt+0x5f/0x80
[677277.993793] [<ffffffffa8aaf122>] locks_remove_file+0x202/0x210
[677278.002847] [<ffffffffc0cd6887>] ? ss_fs_unlocked_ioctl+0x107/0x580 [mmfslinux]
[677278.013661] [<ffffffffa8a5085d>] __fput+0xbd/0x230
[677278.021639] [<ffffffffa8a50abe>] ____fput+0xe/0x10
[677278.029473] [<ffffffffa88c299b>] task_work_run+0xbb/0xe0
[677278.037847] [<ffffffffa882cc65>] do_notify_resume+0xa5/0xc0
[677278.046463] [<ffffffffa8f962ef>] int_signal+0x12/0x17
[677278.054449] ---[ end trace 4f271e130c071185 ]---
...
Environment
- Red Hat Enterprise Linux 7
- IBM gpfs filesystem is in use
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.