RHEL 5 kernel panic when rpcsec_gss_krb5 module is rmmod and then insmod

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 5.4

  • x86, x86_64

Issue

  • kernel panic when re-insertion of rpcsec_gss_krb5 modules
  • kernel panic when rmmod and insmod rpcsec_gss_krb5 module

Unable to handle kernel paging request at 000000000dca2015 RIP:
[<ffffffff883cc0fc>] :sunrpc:auth_domain_put+0x23/0x4d
PGD 16520067 PUD 16b4d067 PMD 0
Oops: 0002 [1] SMP
last sysfs file: /block/dm-1/range
CPU 0
Modules
linked in: rpcsec_gss_krb5 auth_rpcgss autofs4 hidp rfcomm l2cap
bluetooth lockd sunrpc ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables
x_tables ipv6 xfrm_nalgo crypto_api dm_multipath scsi_dh video hwmon
backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac
parport_pc lp parport floppy 8139too virtio_pci 8139cp i2c_piix4 ide_cd
virtio_ring i2c_core virtio mii cdrom serio_raw pcspkr dm_raid45
dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror
dm_log dm_mod ata_piix libata sd_mod scsi_mod ext3 jbd uhci_hcd
ohci_hcd ehci_hcd
Pid: 2657, comm: insmod Not tainted 2.6.18-164.el5 #1
RIP: 0010:[<ffffffff883cc0fc>]  [<ffffffff883cc0fc>] :sunrpc:auth_domain_put+0x23/0x4d
RSP: 0000:ffff810016181ed8  EFLAGS: 00010202
RAX: 12b1801529ba3cc8 RBX: ffff81001ca8b900 RCX: ffff81001ca8b908
RDX: 000000000dca2015 RSI: ffffffff883e5a40 RDI: ffffffff883e5a40
RBP: ffff810016e9f5a0 R08: 73732f6b72623508 R09: ffff810016e9f5a9
R10: 0000000000000000 R11: 0000000000000000 R12: 000000000005f373
R13: ffffffff8849c127 R14: 0000000000000000 R15: ffffffff8848ae3e
FS:  00002b97168ca210(0000) GS:ffffffff803c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000dca2015 CR3: 0000000016912000 CR4: 00000000000006e0
Process insmod (pid: 2657, threadinfo ffff810016180000, task ffff810014bc3100)
Stack:  ffff81001ca8b900 ffffffff88488b1b ffff810016e9f5a0 ffffffff8849d440
ffffffff8849d3a0 ffffffff88487e60 ffff81001d56b040 ffffffff8849d500
00000000127b5030 000000000000bb98 00000000127b5050 0000000000010000
Call Trace:
[<ffffffff88488b1b>] :auth_rpcgss:svcauth_gss_register_pseudoflavor+0x86/0x9c
[<ffffffff88487e60>] :auth_rpcgss:gss_mech_register+0x8e/0x112
[<ffffffff8825f00d>] :rpcsec_gss_krb5:init_kerberos_module+0xd/0x25
[<ffffffff800a5a2e>] sys_init_module+0xaf/0x1f2
[<ffffffff8005d28d>] tracesys+0xd5/0xe0

Code: 48 89 02 74 04 48 89 50 08 48 c7 41 08 00 02 20 00 48 8b 43
RIP  [<ffffffff883cc0fc>] :sunrpc:auth_domain_put+0x23/0x4d
RSP <ffff810016181ed8>

Resolution

  • nfsd-fix-possible-oops-on-re-insertion-of-rpcsec_gss.patch (attached)
  • http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=cb276805803b8e0616159d80a441ab26a931ada4

  • kernel panic when rmmod and insmod rpcsec_gss_krb5 module: https://bugzilla.redhat.com/show_bug.cgi?id=570044
  • Red Hat Enterprise Linux 5.6GA Erratum:
    http://rhn.redhat.com/errata/RHSA-2011-0017.html
    

Root Cause

  • An uninitialised struct hlist_node is operated on resulting in the dereferencing of a random memory location.
  • From the commit summary: "The handling of the re-registration case is wrong here; the "test" that was returned from auth_domain_lookup will not be used again, so that reference should be put.  And auth_domain_lookup never did anything with "new" in this case, so we should just clean it up ourself."

Attachments

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.