OpenJDK "InvalidKeyException: RSA key must be at least 1023 bytes" running in FIPS mode
Issue
- The attached Test.java application runs fine on JDK11 u16.1:
$ java Test
HTTP Response: 200
Successfully connected
But throws the following exception on JDK8 u345:
$ java Test
Connect HTTPS: Unsupported signature algorithm: rsa_pss_rsae_sha256
javax.net.ssl.SSLException: Unsupported signature algorithm: rsa_pss_rsae_sha256
at sun.security.ssl.Alert.createSSLException(Alert.java:133)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.ECDHServerKeyExchange$ECDHServerKeyExchangeMessage.<init>(ECDHServerKeyExchange.java:298)
at sun.security.ssl.ECDHServerKeyExchange$ECDHServerKeyExchangeConsumer.consume(ECDHServerKeyExchange.java:527)
at sun.security.ssl.ServerKeyExchange$ServerKeyExchangeConsumer.consume(ServerKeyExchange.java:122)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422)
at sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:152)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397)
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1572)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1500)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at Test.connectHTTPS(Test.java:52)
at Test.main(Test.java:25)
Caused by: java.security.InvalidKeyException: RSA key must be at least 1023 bytes
Environment
- Red Hat Enterprise Linux in FIPS mode
- Red Hat build of OpenJDK
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.