RHEL6: kernel crash when fips enabled - with NFSD crashes in crypto_larval_kill at list_del

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • fips enabled
  • NFSv4

Issue

  • A VM crashed from a general protection on CPU1. NSFD was in the CPU at the time of the crash.
  • This is the recorded screen dump information:
 general protection fault: 0000 [#1] SMP 
 last sysfs file: /sys/devices/pci0000:00/0000:00:16.0/0000:0b:00.0/local_cpus
 CPU 1 
 Modules linked in: vsock(U) vmci(U) iptable_filter ip_tables nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc ipv6 ipt_REJECT ipt_LOG ppdev parport_pc parport microcode vmware_balloon vmxnet3 i2c_piix4 i2c_core sg shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom mptspi mptscsih mptbase scsi_transport_spi vmw_pvscsi pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmmemctl]

 Pid: 1971, comm: nfsd Not tainted 2.6.32-220.el6.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
 RIP: 0010:[<ffffffff8127b750>]  [<ffffffff8127b750>] list_del+0x10/0xa0
...
 Process nfsd (pid: 1971, threadinfo ffff880233eec000, task ffff88023595b500)
...
 Call Trace:
 [<ffffffff812334b5>] crypto_larval_kill+0x25/0x60
 [<ffffffff81233c3b>] crypto_alg_mod_lookup+0x6b/0x90
 [<ffffffff81233cd1>] crypto_alloc_base+0x41/0xb0
 [<ffffffff81061dc5>] ? dequeue_entity+0x105/0x2e0
 [<ffffffffa0357da6>] nfs4_make_rec_clidname+0x46/0x154 [nfsd]
 [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
 [<ffffffffa03537f3>] nfsd4_setclientid+0xb3/0x4d0 [nfsd]
 [<ffffffff814ef446>] ? _spin_lock_bh+0x16/0x40

Resolution

NOTE Currently, it is not possible to use FIPS mode with NFSv4.

Red Hat Enterprise Linux 6.5

Red Hat Enterprise Linux 6.4 Extended Update Support

Root Cause

  • The crypto_larval_lookup() function could return a larval, an in-between state when a cryptographic algorithm is being registered, even if it did not create one. This could cause a larval to be terminated twice, and result in a kernel panic. This occurred for example when the NFS service was run in FIPS mode, and attempted to use the MD5 hashing algorithm even though FIPS mode has this algorithm blacklisted.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments