openssl enc fails in FIPS mode

Solution Verified - Updated -

Issue

  • Using the openssl enc command to encrypt or decrypt data fails on systems where FIPS is enabled. Example of running it on a normal RHEL machine:

    [user]$ sysctl crypto.fips_enabled
    crypto.fips_enabled = 0
    [user]$ openssl aes-256-cbc -k PASS </etc/redhat-release | openssl aes-256-cbc -d -k PASS
    Red Hat Enterprise Linux Workstation release 6.3 (Santiago)
    

    Here's what happens on a box where the kernel is in FIPS-enforcing mode:

    [user]$ sysctl crypto.fips_enabled
    crypto.fips_enabled = 1
    [user]$ openssl aes-256-cbc -k PASS </etc/redhat-release | openssl aes-256-cbc -d -k PASS
    10283:error:06080090:digital envelope routines:EVP_DigestInit_ex:disabled for fips:digest.c:292:
    bad decrypt
    10284:error:06080090:digital envelope routines:EVP_DigestInit_ex:disabled for fips:digest.c:292:
    10284:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:325:
    ...
    
  • Similar results as above are seen using any other FIPS-approved ciphers with openssl enc (e.g., aes-128-cbc)

Environment

  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In