kernel panic due to openafs filesystem
Issue
- Server was down and customer was not able to connect through ssh or even Virtual machine console also.
-
Server is not booting up, it has a kernel panic.
Process afsd (Pid: 2601, threadinfo ffff8100ca6b2000 Call trace: [<ffffffff884564c7>] :openafs:afs_FlushVcache+0x3fb/0x485 <0>Kernel Panic - not syncing: Fatal exception
- Server panic during boot
openafs: module license 'http://www.openafs.org/dl/license10.html' taints kernel. Found system call table at 0xffffffff80288f40 (pattern scan) Using keyrings, rather than hooking system calls Found 32-bit system call table at 0xffffffff8028a300 (pattern scan) Using keyrings, rather than hooking system calls enabling dynamically allocated vcaches Starting AFS cache scan...found 1681 non-empty cache files (5%). FS-Cache: Loaded usb 2-1: USB disconnect, address 2 osi_linux_free: failed to remove chunk from hashtable afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com osi_linux_free: failed to remove chunk from hashtable osi_linux_free: failed to remove chunk from hashtable afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com <snip> afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com Unable to handle kernel paging request at ffffc2001027a000 RIP: [<ffffffff88816ec7>] :openafs:PGetTokens+0x1a0/0x250 PGD 4080036067 PUD 5080038067 PMD 10b8318067 PTE 0 Oops: 0000 [1] SMP last sysfs file: /devices/pci0000:40/0000:40:03.0/0000:4b:00.0/host5/rport-5:0-2/target5:0:0/5:0:0:4/timeout CPU 39 Modules linked in: nfs fscache nfs_acl openafs(PU) autofs4 ipmi_devintf ipmi_si ipmi_msghandler hidp rfcomm l2cap bluetooth lockd sunrpc cpufreq_ondemand acpi_cpufreq freq_table bonding be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i cxgb3 libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi dm_round_robin dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport joydev sg lpfc pcspkr ide_cd i2c_i801 cdc_ether scsi_transport_fc i2c_core bnx2 usbnet cdrom be2net 8021q dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod cciss ata_piix libata mptsas mptscsih scsi_transport_sas mptbase shpchp megaraid_sas sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd Pid: 35239, comm: aklog Tainted: P 2.6.18-194.11.3.el5 #1 RIP: 0010:[<ffffffff88816ec7>] [<ffffffff88816ec7>] :openafs:PGetTokens+0x1a0/0x250 RSP: 0018:ffff81507b503bb8 EFLAGS: 00010202 RAX: 0000000000001575 RBX: 000000004dac352a RCX: 0000000000001575 RDX: ffffc20010314004 RSI: ffffc2001027a000 RDI: ffffc20010314004 RBP: ffff81507b503c68 R08: ffffc20010314000 R09: 0000000000000004 R10: 0000000000000008 R11: ffffffff888358a0 R12: ffff81105e355a40 R13: 0000000000000001 R14: 0000000000002f44 R15: ffffc20010314000 FS: 00002b9157408760(0000) GS:ffff8140b83d9bc0(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffffc2001027a000 CR3: 00000050794d0000 CR4: 00000000000006e0 Process aklog (pid: 35239, threadinfo ffff81507b502000, task ffff81507e686860) Stack: 0000000000000003 ffffffff800d1596 000000004dabaa12 0000000010a7adef 000000004dabaa12 0000000010a7adef ffffc20010314002 000015750000000a 0000000000000001 0000000000000004 ffff81307a790000 ffffc20010314000 Call Trace: [<ffffffff800d1596>] __vmalloc_area_node+0x12e/0x156 [<ffffffff88813688>] :openafs:afs_HandlePioctl+0x319/0x4a0 [<ffffffff80063b05>] mutex_lock+0xd/0x1d [<ffffffff88815763>] :openafs:afs_syscall_pioctl+0x2a1/0x2cb [<ffffffff88811531>] :openafs:afs_syscall+0x26e/0x1b5a [<ffffffff8000eaaf>] link_path_walk+0xa6/0xb2 [<ffffffff8000f61b>] generic_permission+0x52/0xca [<ffffffff8000d9e4>] permission+0x8d/0xc8 [<ffffffff80012552>] may_open+0x65/0x22f [<ffffffff88807f72>] :openafs:afs_unlocked_ioctl+0x92/0xa0 [<ffffffff80027541>] do_filp_open+0x2a/0x38 [<ffffffff8004214d>] do_ioctl+0x21/0x6b [<ffffffff80030204>] vfs_ioctl+0x457/0x4b9 [<ffffffff800b7605>] audit_syscall_entry+0x180/0x1b3 [<ffffffff8004c633>] sys_ioctl+0x59/0x78 [<ffffffff8005d28d>] tracesys+0xd5/0xe0
Environment
- Red Hat Enterprise Linux 5
- Red Hat Enterprise Linux 6
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.