FIPS mode verification

Latest response

I'm reading about installed RHEL in FIPS mode and I see this:

To fulfil the strict FIPS 140-2 compliance, add the fips=1 kernel option to the kernel command line during system installation.

But then there is also this:

To turn your system, kernel and user space, into FIPS mode anytime after the system installation, follow these steps:

How do I tell which option was used when I am given a system? How do i verify the system is in strict FIPS mode?

Responses

You have two options:

  1. cat /proc/sys/crypto/fips_enabled
  2. sysctl crypto.fips_enabled

1 indicates enabled, while 0 indicates disabled.

Taken from the "Diagnostic Steps" here: https://access.redhat.com/solutions/137833

Correct. But neither of those options show if FIPS was enabled at install or after the fact.

If FIPS is enabled, and you can verify that it is enabled with either method listed above, does it matter how it was enabled?

I'm not 100% sure to be honest. I am just going off the line from the documentation that says "To fulfil the strict FIPS 140-2 compliance, add the fips=1 kernel option to the kernel command line during system installation." to me that implies that turning on FIPS later doesn't fulfil the strict requirements.

If you're "choosing" FIPS due to a mandated compliance requirement (99% of people, I would guess), whenever possible, you should absolutely boot the installer with the "fips=1" argument (instead of doing post-install configuration). Doing so will not only save you time and work (you won't have to do all those steps), but ... well, I've updated that article with a warning at the top which explains.

That is great information. Is there a way to see what might crypto related functions might have been done before FIPS was enabled? For instance is there a way to see if the SSH keys are FIPS compliant?

Is there a way to see what might crypto related functions might have been done before FIPS was enabled?

No... not really. If you're not in full control of the system or are unsure of its history ... that's where it's really recommended to install from scratch with FIPS to ensure all is well.

For instance is there a way to see if the SSH keys are FIPS compliant?

The ssh-keygen command -- usually automatically kicked off by ssh-keygen.service (in modern systemd RHEL) on first-boot -- generates groups for Diffie-Hellman Group Exchange as well as SSH host keys. In modern OpenSSH, I doubt FIPS triggers anything different in the former and as far as the latter: since the keys aren't encrypted, I doubt there's any use of a forbidden hashing algo like md5 -- certainly there's nothing that is permanent, given the fact that you can use ssh host keys generated pre-FIPS on a FIPS system.

The argument is more about the FIPS requirements around RNG testing. Though, ssh-keygen might skip generation of some key-types when you're in FIPS mode, e.g., ed25519.

(That said, password-protected ssh user keys is another story.)

Correct me if I'm wrong, but it sounds like you're reviewing a STIG. I'm sorry for your loss.

To the best of my knowledge, there is no difference. If FIPS is enabled, it is enabled. But it is significantly easier to build a FIPS-compliant system if done during the operation system installation process.

Yes. That is the cross that I bear.

It seems a perfectly reasonable question to me, and actually a reasonable requirement. If the system was installed and only subsequently was FIPS 140-2 applied, then the system could have been subverted in the interval. That is what is implied - to my mind at least - in the phrasing of: "To fulfil the strict FIPS 140-2 compliance, add the fips=1 kernel option to the kernel command line during system installation." I'm not one for "security best practice" - it's usually what some non-technical person has read on the internet - usually something posted by another non-technical person, and is the source IMO of the (bad) advice concerning passwords. However in this case, the original question and reason behind it are both reasonable. Sorry to scold, but I find both answers given to be spectacularly unhelpful. The first didn't read the question, and the second hasn't answered it.

I wrote a script that's based on the solution provided by Red Hat at this discussion post https://access.redhat.com/discussions/3487481

I'm going to add more sanity clauses into the script soon, but it's pretty good now.

Regards,

RJ

Brian asked above

If FIPS is enabled, and you can verify that it is enabled 
with either method listed above, does it matter how it 
was enabled?
  • the answer is a tentative "yes" because even after I successfully implemented FIPS using the solution (and a script I wrote to implement the solution, cited above), the STIG check from DISA still failed, even though it was running. So I had to dig into the complaints of the STIG checker and found that the complaints (in this case) wanted a "fips=1" for EVERY instance of a kernel in /boot/grub2/grub.cfg, and if that wasn't enough, it wanted an additional entry in the /etc/defaults/grub for GRUB_CMDLINE_LINUX_DEFAULT="fips=1" or it would fail too. I had already put a value in for GRUB_CMDLINE_LINUX (see my script, previous post) but that was not sufficient to make it go away in the stig check. So yes, it worked, but it "failed" grrrrr due to the broken nature of the stig checker, but it could be argued the configs were necessary for subsequent kernels (even though my original script fulfilled that.

The alternative is to explain to a disinterested person why your work is correct but the stig check fails. Now in some cases this is going to happen. I've taken additional steps (like in this specific case) to make sure both pass.

enjoy

begin ADDED: See Tom's post for additional clarity. I wrote this with the intention that if you happen to have to carry the cross of STIG compliance, there are additional steps than some of the initial steps for some items to get the compliance software to register. That being said, some of the "checks" are dumb on the compliance check software and it will issue "false positives". There are some things that may be worth it to your organization to "POA&M" and not fulfill compliance on.. There are some STIG things we don't do on specific servers for that reason. Some servers can take most of the security settings we need, some take less based on their server role and if we can or can not implement a given STIG rule (such as FIPS). One example, FIPS and Satellite server, or Gluster does not go together.

end added

Spent a lot of years working in the "STIG required" space: have yet to see a system that is able to get a 100% mark on a compliance-scan ...except in the vanishingly-rare instance where the accreditor is using a scan-profile that's been properly aligned to the specifics of the deployment-environment and the workload the system will be hosting. In a complex, "enterprise" environment - where you may have some systems being hosted on a variety of physical hardware, virtualization frameworks and "cloud"solutions, the typical scores tend to group-out environment-by-environment: e.g., your accreditor's scan-profile typically says "DHCP must be disabled" but your "cloud" hosted systems can never pass this test due to underlying requirements of your cloud service.

In general, if you've taken the proper dev/test/prod deployment cycle, you should know before your systems leave dev for test what scan-items the eventual prod systems will "fail". Having this all documented - and part of your CCP document-bundle - is frequently sufficient for semi-clueful accreditors.

Except that FIPS mode is a CAT1. No getting around it.

Given that there's a number of COTS products that simply will not operate under FIPS mode, you can get around it via your POA&M filing. Which, again, is something you should know to have documented well before your official accreditor has sat down to run his tools against it.

Absolutely, if FIPS (or some STIG thing) breaks the function of something, and it's important enough, it can be dealt with as Tom says. The same with many other things that are in the STIG. We don't take "everything" either, and some things we "POA&M" That being said, our stig compliance is nicely high, while still being functional. A system that gets 100% is probably not useful at all. Ours are not 100% and I too have yet to see a system that is 100% just as Tom said.

My intent is simply to answer that one itch that Brian asked, not to make some statement that 100% to be the goal because that is totally unrealistic.

I apologize. I guess my general feelings for what can seem like vague security rules got in the way of helping.

Good to know, R. Hinton! Thanks for digging into this.

Brian, this is just a discussion, and place to learn, so no need for any apologies. Just discussion options. Tom brings up some good points and some context (as he usually does).

In case anyone stumbles on this thread when searching for information on enabling fips mode on RHEL 7, I learned that if you set "fips=1" on the system installer kernel, that the dracut-fips package will be installed automatically and the UUID parameter is automatically set on the kernel boot line. For my custom kickstart, I needed to update the file "isolinux.cfg" and update the kernel lines in that file to include "fips=1". I learned that setting "fips=1" on the the kickstart "bootloader --append" command does not affect the kernel for the system installer, but only for the kernel after the system is installed. This was the key thing I learned that may not be obvious to everyone.

And also see the script I wrote/cited earlier. And see Ryan Sawhill's post{s) above. My script adds the proper uuid.

Are you saying the UUID the installer adds automatically (assuming you've booted the installer with "fips=1") is not the proper UUID?

Hi Jonathan,

What I'm saying is my script automatically populates the uuid for boot properly in the /etc/default/grub and /boot/grub/grub.cfg (or uuid version) correctly. Check the link for the script I cited earlier, it is all in there.

Regards

RJ

Hi RJ, I checked out your script and agree, it's definitely worth using if one needs to enable FIPS mode after you've already installed RHEL 7. My point is that the RHEL 7 system installer will automatically install dracut-fips and setup UUID properly (no extra script necessary) if you set "fips=1" on the system installer's kernel boot parameters. Thanks for the feedback,

-Will

Will, I didn't know that, thanks for the good info!

-RJ

The RHEL 8 beta installer automatically enabled FIPS and now autofs will not mount my Samba NAS shares (manually mounting fails as well). I've found that Samba needs MD5 which is disabled when 'sysctl crypto.fips_enabled' returns 'crypto.fips_enabled = 1'.

Now for test I manually removed the 'fips=1' directive from the kernel line during boot, and sure enough, I now get 'crypto.fips_enabled = 0'. No complaints anymore about cyphers in /var/log/messages, but mount.cifs still fails, this time with CIFS VFS: BAD_NETWORK_NAME: \\[server]\[share], and I can't seem to get around that.

This worked (and works still) flawlessly under CentOS 5.11, 6.10 and 7.5, which do not have fips enabled. Is there anything else to disabling FIPS, or is there a trick (preferred!) to make SMB mounting work with fips enabled?

RHEL 8 is still beta. Would probably be a good idea of you to look for an existing bug-ID and file a bug if no relevant bug-IDs exist.

Thanks, but I 'm not sure it's a bug yet, as fips is known to break SMB mounts on RHEL 7 as well. I'll do some more research, probably need to break a CentOS 7 VM as well for verification :-).

From the standpoint that (you seem to be indicating) RHEL 8 is defaulting to FIPS mode, it'd still be worth reporting.

Thanks, I'll throw it into the RHEL 8 beta thread, see what the opinion is. I might actually have induced it by selecting one of the security profiles during installation. Simple test by creating another VM without it. Probably best to try that first. I might have hardened more than I can chew ;-)

At the very least, it probably deserves a documentation-note or, in a GUI or menu-driven installer, a warning-bubble (or equivalent). =)

Built a new VM without selecting one of the security profiles and all works as expected. So in the end this is no different from RHEL 7, where I also never selected this. The second profile mentiones something about mandatory for government etc., iirc, so I gues it's to be expected that FIPS will be enabled.

Just me trying to do too much in one go. Thanks for your responses and sorry for creating confusion, if any.

Unfortunately, even after setting "update-crypto-policy --set DEFAULT", (on RHEL8.2) it's still impossible to enable Samba shares. Perhaps there's another setting somewhere that needs to be toggled, but it's not clear to me what that setting is.

Does anyone know of a way to set fips=0 without rebooting and changing the value via kernel? Essentially I just need to disable fips momentarily to mount an SMB, then return to fips=1 after SMB has been unmounted and files I need have been moved off the machine.

As with SELinux, FIPS-mode is a kernel-level setting. Effectively, if you want to disable FIPS-mode, you have to boot to a different kernel.

This is not an absolute answer: but, if I was looking in to this, I would review /root/anaconda-ks.cfg - more specifically, I would run

# grep ^bootl /root/anaconda-ks.cfg 
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda

Also - if you look at the content in /boot - and see a set of non-FIPS files in there, I believe one could deduce it had been converted and not built originally with FIPS enabled. (I am going to build some stuff in my lab to confirm).

EDIT: my response is only in regards to whether you could figure out of FIPS was from build time, or enabled post-install.

Inherent problem with this method is that the anaconda-ks.cfg file may not exist on a system (for any number of reasons). I know that it doesn't exist on any of our monthly cloud templates, since we don't don't use anaconda to build systems. Depending on your security requirements, you might have your build-automation either not-create or clean up the anaconda-ks.cfg file since it notionally contains sensitive information (I know our IA teams got pissy when their scan tools discovered that our default hardening-automation deposited a courtesy 000600-set scan-report in root's 000700-set home-directory)

Thanks Thomas! I had no idea that was an option (to not create an anaconda file). TIL!

Hey James.

By the way, /root/anaconda-ks.cfg is, of course, the resultant of the build process, and has zero impact on the boot process. It might be good to interrogate the anaconda-ks.cfg file to see how the build process went, but bear that in mind.

The script I wrote (and mentioned above) does all the work of the Red Hat solution to include validating /etc/default/grub and all the necessary directives in /boot/grub2/grub.cfg

Anyone wanting to verify if FIPS is running, make sure the dracut-fips rpm is installed and also run (as root) sysctl crypto.fips_enabled. If dracut-fips is not installed, then FIPS even if in grub is not fully active.

Regards,
RJ

Hey - it's been a long time ;-) I was only responding to the ask of whether it was enabled from "build" or "after the build". I am trying to figure out if running stat on /etc/system-fips might also shed some light - but... in my limited research, I just found that my newly built box has an mtime of 2019-06-19 - so, I assume the fudge that date for a reason?

Oh, it's tremendously easier to do it from the build. That being said, I'm rereading what Ryan said, his method at install time seems easier than my method of using my script at install time (even though my script works great).

You say your time has a mtime of 2019-06-19? Do you just need to enable ntpd, chronyd or exercise some ntpdate commands?

Hope you're doing well,
RJ

No - it's a VM and the date/time is accurate (weird, right?) I don't really care much - just made me curious ;-)

 File: ‘/etc/system-fips’
  Size: 0           Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 424218      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:etc_t:s0
Access: 2020-03-09 19:20:46.855561050 -0500
Modify: 2019-06-19 08:05:03.000000000 -0500
Change: 2020-03-09 19:20:11.380126036 -0500

The Modify field is essentially meaningless. The date you're seeing is the date that the RPM for that file was created:

$ stat /etc/system-fips
  File: ‘/etc/system-fips’
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 139067      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:etc_t:s0
Access: 2020-02-25 16:15:24.833442235 +0000
Modify: 2019-08-08 23:14:24.000000000 +0000
Change: 2019-12-17 14:00:59.011589096 +0000
 Birth: -
$ rpm -qi $( rpm -qf /etc/system-fips )
Name        : dracut-fips
Version     : 033
Release     : 564.el7
Architecture: x86_64
Install Date: Tue 17 Dec 2019 02:00:59 PM UTC
Group       : System Environment/Base
Size        : 8214
License     : GPLv2+ and LGPLv2+
Signature   : RSA/SHA256, Thu 22 Aug 2019 09:22:28 PM UTC, Key ID 24c6a8a7f4a80eb5
Source RPM  : dracut-033-564.el7.src.rpm
Build Date  : Thu 08 Aug 2019 11:14:34 PM UTC
Build Host  : x86-02.bsys.centos.org
Relocations : (not relocatable)
Packager    : CentOS BuildSystem <http://bugs.centos.org>
Vendor      : CentOS
URL         : https://dracut.wiki.kernel.org/
Summary     : dracut modules to build a dracut initramfs with an integrity check
Description :
This package requires everything which is needed to build an
initramfs with dracut, which does an integrity check.

The information in the above is taken from is a VM whose template wasn't even created until December (which aligns to the Change date in the stat output) and launched a few weeks later.

I ended up here attempting to get the FIPS control properly passing for a STIG on RHEL7. After quite a bit of searching and work, I found a missing detail that allowed the scan to Pass. Installing 'dracut-fips-aesni' finally resolved the failing scan. Original reference here: https://access.redhat.com/solutions/137833