Satellite 6: candlepin and candlepin_auth fail with response "404 Resource Not Found" and "Katello::Resources::Candlepin::CandlepinPing: 404 Resource Not Found" (SSLEngine problem)
Environment
- Red Hat Satellite 6
Issue
- candlepin and candlepin_auth fail on the Satellite server
# hammer ping
candlepin:
Status: FAIL
Server Response: Message: 404 Resource Not Found
candlepin_auth:
Status: FAIL
Server Response: Message: Katello::Resources::Candlepin::CandlepinPing: 404 Resource Not Found (GET /candlepin/status)
Resolution
Satellite 6.6 and newer
- Remove all Candlepin certificates:
# rm -f /etc/candlepin/certs/keystore
# rm -f /etc/candlepin/certs/amqp/*
- Remove files in /etc/pki/katello/nssdb directory:
# rm -rf /etc/pki/katello/nssdb/*
- Run satellite-installer
# satellite-installer --scenario satellite
Satellite 6.5 and older
- Remove all Candlepin certificates:
# rm -f /etc/pki/katello/keystore
# rm -f /etc/candlepin/certs/amqp/*
- Remove files in /etc/pki/katello/nssdb directory:
# rm -rf /etc/pki/katello/nssdb/*
- Run satellite-installer
# satellite-installer --scenario satellite
For more KB articles/solutions related to Red Hat Satellite 6.x Candlepin Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x Candlepin Issues
Root Cause
- Candlepin certificates are not signed by the current certification authority "/etc/pki/katello/certs/katello-default-ca.crt"
Candlepin component:
/etc/candlepin/certs/amqp/candlepin.truststore
/etc/candlepin/certs/amqp/candlepin.jks
Tomcat:
Satellite 6.5 and older
/etc/pki/katello/keystore
Satellite 6.6. and newer
/etc/candlepin/certs/keystore
Diagnostic Steps
"hammer ping" shows the following status:
# hammer ping
candlepin:
Status: FAIL
Server Response: Message: 404 Resource Not Found
candlepin_auth:
Status: FAIL
Server Response: Message: Katello::Resources::Candlepin::CandlepinPing: 404 Resource Not Found (GET /candlepin/status)
pulp:
Status: ok
Server Response: Duration: 32ms
pulp_auth:
Status: ok
Server Response: Duration: 16ms
elasticsearch:
Status: ok
Server Response: Duration: 16ms
foreman_tasks:
Status: ok
Server Response: Duration: 0ms
Discrepancy in "keyid" between certification authority "/etc/pki/katello/certs/katello-default-ca.crt" and "keystore":
# openssl x509 -text -in katello-default-ca.crt |grep keyid
keyid:2A:75:E8:8B:21:43:A1:39:B5:C4:CB:6D:51:0D:1F:53:B6:0A:F6:BF
# keytool -list -v -keystore /etc/candlepin/certs/amqp/candlepin.truststore
...
#2: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 2A 75 E8 8B 21 43 A1 39 B5 C4 CB 6D 51 0D 1F 53 *u..!C.9...mQ..S
0010: B6 0A F6 BF
...
# keytool -v -list -keystore /etc/pki/katello/keystore --storetype PKCS12 -storepass $(sed -e '/keystorePass/!d' /etc/tomcat/server.xml -e 's/\s*keystorePass=//' -e "s/\"//g")
...
#2: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 2A 75 E8 8B 21 43 A1 39 B5 C4 CB 6D 51 0D 1F 53 *u..!C.9...mQ..S
0010: B6 0A F6 BF
...
The following messages are logged to /var/log/candlepin/candlepin.log:
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1348) ~[na:1.7.0_99]
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:519) ~[na:1.7.0_99]
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1200) ~[na:1.7.0_99]
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1172) ~[na:1.7.0_99]
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) ~[na:1.7.0_99]
at org.apache.qpid.transport.network.security.ssl.SSLSender.send(SSLSender.java:157) ~[qpid-common-0.30.redhat-1.jar:0.30.redhat-1]
... 48 common frames omitted
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.7.0_99]
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1714) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:281) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273) ~[na:1.7.0_99]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1472) ~[na:1.7.0_99]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:213) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:913) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker$1.run(Handshaker.java:853) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker$1.run(Handshaker.java:851) ~[na:1.7.0_99]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_99]
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1285) ~[na:1.7.0_99]
at org.apache.qpid.transport.network.security.ssl.SSLReceiver.doTasks(SSLReceiver.java:206) ~[qpid-common-0.30.redhat-1.jar:0.30.redhat-1]
at org.apache.qpid.transport.network.security.ssl.SSLReceiver.received(SSLReceiver.java:165) ~[qpid-common-0.30.redhat-1.jar:0.30.redhat-1]
at org.apache.qpid.transport.network.security.ssl.SSLReceiver.received(SSLReceiver.java:36) ~[qpid-common-0.30.redhat-1.jar:0.30.redhat-1]
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:161) ~[qpid-common-0.30.redhat-1.jar:0.30.redhat-1]
... 1 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:350) ~[na:1.7.0_99]
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:260) ~[na:1.7.0_99]
at sun.security.validator.Validator.validate(Validator.java:260) ~[na:1.7.0_99]
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326) ~[na:1.7.0_99]
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:283) ~[na:1.7.0_99]
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:138) ~[na:1.7.0_99]
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1459) ~[na:1.7.0_99]
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments