HMAC SHA 256 authentication error in Java using FIPS mode

Solution Unverified - Updated -

Environment

  • OpenJDK 17.0.4 in RHEL 8 container running in FIPS mode.

Issue

  • When tried to open a TLS connection using OpenJDK 17.0.4 in RHEL 8 container running in FIPS mode the exception [1] is observed .
  • While using HmacSHA256 (SHA2) crypto algorithm getting authentication error [1] in OpenJDK 17.0.4 in RHEL 8 using FIPS mode.

[1]

Caused by: com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=SCRAM-SHA-256, userName='abcdef', source='ghijk', password=<hidden>, mechanismProperties=<hidden>}
Caused by: javax.security.sasl.SaslException: Invalid key for HmacSHA256
Caused by: java.security.InvalidKeyException: init() failed
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_SIZE_RANGE

Resolution

  • If you do not want FIPS to be enabled in java , you can disable it using the jvm options "-Dcom.redhat.fips=false" or set security.useSystemPropertiesFile=false in java.security see [1][2]

  • If you wish to keep the FIPS enabled and still observed the issue then it caused due to short password for the username. In current scenario the password for the user "abcdef" should be atleast 16 byte long.

[1] How to disable FIPS for java when FIPS is enabled on RHEL 8?

[2] How do I make JBoss EAP/Tomcat use FIPS 140-2 compliant cryptography?

Root Cause

  • CKR_KEY_SIZE_RANGE means that the key passed to calculate the hash-based message authentication code (HMAC) is too short for the restrictions imposed by FIPS. The key must be at least 16 bytes long.

Diagnostic Steps

  • Check if FIPS is enabled or not using :
    fips-mode-setup --check

  • Check if the crypto algorithm is supported in current java release

#cat KeyGeneratorTest.java
import javax.crypto.KeyGenerator;
public class KeyGeneratorTest {
        public static void main(String[] args) throws Exception {
                KeyGenerator kg = KeyGenerator.getInstance("HmacSHA256");
                System.out.println(kg);
                System.out.println("KeyGenerator provider: " + kg.getProvider().getName());
        }
}


#/usr/lib/jvm/java-17-openjdk/bin/javac KeyGeneratorTest.java

#/usr/lib/jvm/java-17-openjdk/bin/java KeyGeneratorTest
javax.crypto.KeyGenerator@5f4da5c3
KeyGenerator provider: SunPKCS11-NSS-FIPS

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments