Issue

• Thumbslug fails at service start with the following error in its error.log /var/log/thumbslug/error.log -

$ katello-server start
...
Mar 15 11:16:04 [main] WARN  org.candlepin.thumbslug.Main - Shutting down...
Mar 15 11:16:05 [main] INFO  org.candlepin.thumbslug.ssl.SslContextFactory - reading keystore
Mar 15 11:16:07 [main] WARN  org.candlepin.thumbslug.Main - Running Thumbslug on port 8088

• Client system registered to the SAM server encounters error such as the following:

$ yum install emacs
Loaded plugins: product-id, security, subscription-manager
Updating certificate-based repositories.
https://upgradetest9:8088/content/dist/rhel/server/6/6Server/x86_64/cf-tools/1/os/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 502"
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhel-6-server-cf-tools-1-rpms. Please verify its path and try again

...with corresponding error in Thumbslug error log /var/log/thumbslug/error.log:

Mar 15 11:27:27 [New I/O client worker #1-1] ERROR org.candlepin.thumbslug.ssl.SslContextFactory - Unable to load pem file!
java.io.IOException: Invalid PEM file: Unknown marker for private key -----BEGIN ENTITLEMENT DATA-----
        at org.candlepin.thumbslug.ssl.PEMx509KeyManager.getPrivateKeyFromPem(PEMx509KeyManager.java:83)
        at org.candlepin.thumbslug.ssl.PEMx509KeyManager.addPEM(PEMx509KeyManager.java:60)
        at org.candlepin.thumbslug.ssl.SslContextFactory.getClientContext(SslContextFactory.java:135)
        at org.candlepin.thumbslug.HttpCdnClientChannelFactory.buildFinalPipeline(HttpCdnClientChannelFactory.java:68)
        at org.candlepin.thumbslug.HttpCdnClientChannelFactory.getPipeline(HttpCdnClientChannelFactory.java:107)
        at org.candlepin.thumbslug.HttpRequestHandler.beginCdnCommunication(HttpRequestHandler.java:251)
        at org.candlepin.thumbslug.HttpRequestHandler.access$100(HttpRequestHandler.java:56)
        at org.candlepin.thumbslug.HttpRequestHandler$1.onResponse(HttpRequestHandler.java:153)
        at org.candlepin.thumbslug.HttpCandlepinClient$HttpResponseHandler.messageReceived(HttpCandlepinClient.java:128)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
        at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:158)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
        at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:131)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndFireMessageReceived(ReplayingDecoder.java:515)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:506)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.cleanup(ReplayingDecoder.java:541)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.channelDisconnected(ReplayingDecoder.java:449)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
        at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:77)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:344)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:226)
        at org.jboss.netty.handler.ssl.SslHandler.channelDisconnected(SslHandler.java:477)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
        at org.jboss.netty.channel.Channels.fireChannelDisconnected(Channels.java:360)
        at org.jboss.netty.channel.socket.nio.NioWorker.close(NioWorker.java:595)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:358)
        at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:679)