kernel panic due to openafs filesystem

Solution Verified - Updated -

Issue

  • Server was down and customer was not able to connect through ssh or even Virtual machine console also.
  • Server is not booting up, it has a kernel panic.

    Process afsd (Pid: 2601, threadinfo ffff8100ca6b2000 
    Call trace:
    [<ffffffff884564c7>] :openafs:afs_FlushVcache+0x3fb/0x485
    <0>Kernel Panic - not syncing: Fatal exception
    
  • Server panic during boot
    openafs: module license 'http://www.openafs.org/dl/license10.html' taints kernel.
    Found system call table at 0xffffffff80288f40 (pattern scan)
    Using keyrings, rather than hooking system calls
    Found 32-bit system call table at 0xffffffff8028a300 (pattern scan)
    Using keyrings, rather than hooking system calls
    enabling dynamically allocated vcaches
    Starting AFS cache scan...found 1681 non-empty cache files (5%).
    FS-Cache: Loaded
    usb 2-1: USB disconnect, address 2
    osi_linux_free: failed to remove chunk from hashtable
    afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com
    osi_linux_free: failed to remove chunk from hashtable
    osi_linux_free: failed to remove chunk from hashtable
    afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com
    <snip>
    afs: Waiting for busy volume 536870957 (user.bakkeg) in cell hostname.example.com
    Unable to handle kernel paging request at ffffc2001027a000 RIP: 
     [<ffffffff88816ec7>] :openafs:PGetTokens+0x1a0/0x250
    PGD 4080036067 PUD 5080038067 PMD 10b8318067 PTE 0
    Oops: 0000 [1] SMP 
    last sysfs file: /devices/pci0000:40/0000:40:03.0/0000:4b:00.0/host5/rport-5:0-2/target5:0:0/5:0:0:4/timeout
    CPU 39 
    Modules linked in: nfs fscache nfs_acl openafs(PU) autofs4 ipmi_devintf ipmi_si ipmi_msghandler hidp rfcomm l2cap bluetooth lockd sunrpc cpufreq_ondemand acpi_cpufreq freq_table bonding be2iscsi ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i cxgb3 libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi dm_round_robin dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport joydev sg lpfc pcspkr ide_cd i2c_i801 cdc_ether scsi_transport_fc i2c_core bnx2 usbnet cdrom be2net 8021q dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod cciss ata_piix libata mptsas mptscsih scsi_transport_sas mptbase shpchp megaraid_sas sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
    Pid: 35239, comm: aklog Tainted: P      2.6.18-194.11.3.el5 #1
    RIP: 0010:[<ffffffff88816ec7>]  [<ffffffff88816ec7>] :openafs:PGetTokens+0x1a0/0x250
    RSP: 0018:ffff81507b503bb8  EFLAGS: 00010202
    RAX: 0000000000001575 RBX: 000000004dac352a RCX: 0000000000001575
    RDX: ffffc20010314004 RSI: ffffc2001027a000 RDI: ffffc20010314004
    RBP: ffff81507b503c68 R08: ffffc20010314000 R09: 0000000000000004
    R10: 0000000000000008 R11: ffffffff888358a0 R12: ffff81105e355a40
    R13: 0000000000000001 R14: 0000000000002f44 R15: ffffc20010314000
    FS:  00002b9157408760(0000) GS:ffff8140b83d9bc0(0000) knlGS:0000000000000000
    CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: ffffc2001027a000 CR3: 00000050794d0000 CR4: 00000000000006e0
    Process aklog (pid: 35239, threadinfo ffff81507b502000, task ffff81507e686860)
    Stack:  0000000000000003 ffffffff800d1596 000000004dabaa12 0000000010a7adef
     000000004dabaa12 0000000010a7adef ffffc20010314002 000015750000000a
     0000000000000001 0000000000000004 ffff81307a790000 ffffc20010314000
    Call Trace:
     [<ffffffff800d1596>] __vmalloc_area_node+0x12e/0x156
     [<ffffffff88813688>] :openafs:afs_HandlePioctl+0x319/0x4a0
     [<ffffffff80063b05>] mutex_lock+0xd/0x1d
     [<ffffffff88815763>] :openafs:afs_syscall_pioctl+0x2a1/0x2cb
     [<ffffffff88811531>] :openafs:afs_syscall+0x26e/0x1b5a
     [<ffffffff8000eaaf>] link_path_walk+0xa6/0xb2
     [<ffffffff8000f61b>] generic_permission+0x52/0xca
     [<ffffffff8000d9e4>] permission+0x8d/0xc8
     [<ffffffff80012552>] may_open+0x65/0x22f
     [<ffffffff88807f72>] :openafs:afs_unlocked_ioctl+0x92/0xa0
     [<ffffffff80027541>] do_filp_open+0x2a/0x38
     [<ffffffff8004214d>] do_ioctl+0x21/0x6b
     [<ffffffff80030204>] vfs_ioctl+0x457/0x4b9
     [<ffffffff800b7605>] audit_syscall_entry+0x180/0x1b3
     [<ffffffff8004c633>] sys_ioctl+0x59/0x78
     [<ffffffff8005d28d>] tracesys+0xd5/0xe0
    

Environment

  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.