RHEL5: Kernel crash in shrink_dcache_for_unmount while unmounting 3rd party file system
Issue
- The first kernel errata for RHEL 5.8 (2.6.18-308.1.1) seems to have introduced an issue where a third party file system (PVFS / OrangeFS) can no longer be unmounted without triggering a kernel panic.
- The initial 5.8 release (2.6.18-308) and previous RHEL releases did not produce this issue, but each errata release in the 5.8 sequence - through 2.6.18-308.24.1 - does produce a panic.
- The panic can be triggered by mounting a PVFS file system, executing a dd that moves data onto the file system (50MB from /dev/urandom in my case), and then unmounting the file system.
- Kernel crashes with a message similar to the following, which shows that the kernel is tainted, and a 3rd party fileystem is being used
pvfs2: pvfs2_init called with debug mask: "none" (0x00000000)
pvfs2: module version 2.8.6-orangefs-2012-12-17-194734 loaded
PVFS: kernel debug mask has been modified to "none" (0x00000000)
PVFS: client debug mask has been modified to "none" (0x00000000)
pvfs2: module version 2.8.2 loaded
BUG: Dentry ffff8100205b7c48{i=100002,n=batayl} still in use (-1) [unmount of pvfs28 pvfs28]
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at fs/dcache.c:612
invalid opcode: 0000 [1] SMP
last sysfs file: /devices/pci0000:00/0000:00:00.0/irq
CPU 1
Modules linked in: nfs nfs_acl pvfs28(U) orangefs(U) autofs4 hidp l2cap bluetooth lockd sunrpc be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i libcxgbi cxgb3 8021q libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi vsock(U) vmmemctl(U) acpiphp dm_mirror dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac mptctl lp vmci(U) sg shpchp floppy ide_cd i2c_piix4 i2c_core cdrom parport_pc parport tpm_tis tpm tpm_bios pcspkr serio_raw vmxnet(U) pvscsi(U) vmxnet3(U) dm_raid45 dm_message dm_region_hash dm_log dm_mod dm_mem_cache ata_piix libata mptspi mptscsih mptbase scsi_transport_spi sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 4582, comm: umount Tainted: G ---- 2.6.18-308.24.1.el5 #1
RIP: 0010:[<ffffffff800effd3>] [<ffffffff800effd3>] shrink_dcache_for_umount_subtree+0x177/0x222
RSP: 0018:ffff810020607e08 EFLAGS: 00010286
RAX: 0000000000000060 RBX: ffff8100205b7c48 RCX: ffffffff80324028
RDX: ffffffff80324028 RSI: 0000000000000000 RDI: ffffffff80324020
RBP: ffff810020808238 R08: ffffffff80324028 R09: 000000000000003f
R10: ffff810020607aa8 R11: 0000000000000080 R12: ffff8100205b7ca8
R13: 0000000000000000 R14: 0000000000000000 R15: 00000000072f9580
FS: 00002b0211862780(0000) GS:ffff81003fe1c7c0(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003d860e7940 CR3: 000000002061a000 CR4: 00000000000006a0
Process umount (pid: 4582, threadinfo ffff810020606000, task ffff81003c8be040)
Stack: ffff81003ee04220 0000000000000206 ffff81003ee04000 ffffffff88716280
ffff81003ee040d0 ffffffff800f04ef ffff81003ee04000 ffffffff800e7850
000000000000001c ffff81003ee04000 ffff81003ee040d0 ffffffff800e797e
Call Trace:
[<ffffffff800f04ef>] shrink_dcache_for_umount+0x37/0x45
[<ffffffff800e7850>] generic_shutdown_super+0x1b/0xfb
[<ffffffff800e797e>] kill_anon_super+0x9/0x35
[<ffffffff88704476>] :pvfs28:pvfs2_kill_sb+0xe9/0x173
[<ffffffff800e7a2f>] deactivate_super+0x6a/0x82
[<ffffffff800f28f7>] sys_umount+0x245/0x27b
[<ffffffff800237fa>] sys_newstat+0x19/0x31
[<ffffffff8009b025>] recalc_sigpending+0xe/0x25
[<ffffffff8001de35>] sigprocmask+0xb7/0xdb
[<ffffffff80030239>] sys_rt_sigprocmask+0x50/0xd9
[<ffffffff8005d116>] system_call+0x7e/0x83
Code: 0f 0b 68 27 2a 2c 80 c2 64 02 5f 41 58 eb fe 48 8b 6b 28 48
RIP [<ffffffff800effd3>] shrink_dcache_for_umount_subtree+0x177/0x222
RSP <ffff810020607e08>
Environment
- Red Hat Enterprise Linux 5
- Kernels 2.6.18-308.1.1 or above
- 3rd party filesystem such as PVFS, VxFS, or Orangefs
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.