JBoss EAP 7.x Management Console in Domain Mode Intermittent Connection Interface Problem

Issue

• Recently HAL has started failing to load as it should and fails back to connection prompt
• It started intermittently then stopped working completely over the course of several weeks, although jboss-cli and deployed applications still work
• 500 (Internal Server Error) responses trying to access JBoss EAP management console

• Broken pipe errors in the logs intermittently related to the problem:

  09:01:22,873 WARN  [org.xnio] (management I/O-1) XNIO000307: Failed to close ssl engine when handling exception java.io.IOException: Broken pipe: java.io.IOException: Broken pipe
      at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.8.0_332]
      at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1.8.0_332]
      at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) [rt.jar:1.8.0_332]
      at sun.nio.ch.IOUtil.write(IOUtil.java:51) [rt.jar:1.8.0_332]
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) [rt.jar:1.8.0_332]
      at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:153)
      at org.xnio.ssl.JsseSslConduitEngine.doFlush(JsseSslConduitEngine.java:851) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.ssl.JsseSslConduitEngine.handleWrapResult(JsseSslConduitEngine.java:410) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.ssl.JsseSslConduitEngine.wrapCloseMessage(JsseSslConduitEngine.java:823) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.ssl.JsseSslConduitEngine.closeOutbound(JsseSslConduitEngine.java:904) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.ssl.JsseSslStreamSinkConduit.terminateWrites(JsseSslStreamSinkConduit.java:136) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.conduits.ConduitStreamSinkChannel.shutdownWrites(ConduitStreamSinkChannel.java:178) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.jboss.remoting3.remote.RemoteConnection$RemoteWriteListener.doShutdownWrites(RemoteConnection.java:356) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.jboss.remoting3.remote.RemoteConnection$RemoteWriteListener.shutdownWrites(RemoteConnection.java:347) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.jboss.remoting3.remote.RemoteConnection.shutdownWrites(RemoteConnection.java:135) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.jboss.remoting3.remote.RemoteConnectionHandler.receiveCloseRequest(RemoteConnectionHandler.java:229) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:97) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.jboss.remoting3.remote.RemoteReadListener.handleEvent(RemoteReadListener.java:51) [jboss-remoting-5.0.23.SP1-redhat-00001.jar:5.0.23.SP1-redhat-00001]
      at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66) [xnio-api-3.8.6.Final-redhat-00001.jar:3.8.6.Final-redhat-00001]
      at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
      at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)