CLI multiple :start() fail after a deploy in EAP 6
Issue
-we are using a ruby script to start in parallel 15 JBoss instances on a domain configuration.
We have 1 domain controller and 15 host controller machines each with 1 JBoss instance. All instances are owned by a single server group.
The script creates 15 threads each calling jboss-cli.sh to launch at the same a :start(blocking=true) on each instance. We do not use the server group :start() operation because it's not synchronous.
The problem:
We found that after a deploy (managed), the ruby script fails because many threads "jboss-cli.sh" is failing with this exception:
org.jboss.as.cli.CliInitializationException: Failed to connect to the controller
at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:280)
at org.jboss.as.cli.impl.CliLauncher.main(CliLauncher.java:246)
at org.jboss.as.cli.CommandLineMain.main(CommandLineMain.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.modules.Module.run(Module.java:270)
at org.jboss.modules.Main.main(Main.java:411)
Caused by: org.jboss.as.cli.CommandLineException: The controller is not available at localhost:9999
at org.jboss.as.cli.impl.CommandContextImpl.tryConnection(CommandContextImpl.java:951)
at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandContextImpl.java:790)
at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandContextImpl.java:766)
at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:278)
... 8 more
Caused by: java.io.IOException: java.net.ConnectException: JBAS012144: Could not connect to remote://localhost:9999. The connection timed out
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:129)
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:71)
at org.jboss.as.cli.impl.CommandContextImpl.tryConnection(CommandContextImpl.java:929)
... 11 more
Caused by: java.net.ConnectException: JBAS012144: Could not connect to remote://localhost:9999. The connection timed out
at org.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:131)
at org.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:256)
at org.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:70)
at org.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:176)
at org.jboss.as.cli.impl.CLIModelControllerClient.getOrCreateChannel(CLIModelControllerClient.java:155)
at org.jboss.as.cli.impl.CLIModelControllerClient$2.getChannel(CLIModelControllerClient.java:117)
at org.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:115)
at org.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:90)
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:236)
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:141)
at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:127)
... 13 more
We are guessing that DC, for some reason, is temporarily refusing connections from jboss-cli.sh instances.
Note that the DC does not need to be restarted: later, if we start the same ruby script (with same parallelism) the start finish successfully.
- is really DC limiting connections or is it a bug related to the deploy?
- is there a workaround to start all the 15 instances at the same time in a synchronized manner?
Environment
- Redhat Enterprise Application Platform (EAP) 6.1.0
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
