Second BxMS server fails to start due to java.util.NoSuchElementException in a Zookeeper/Helix clustered environment

Solution Verified - Updated -

Issue

We are trying to get a clustered BxMS with Zookeeper/Helix running on two different physical machines. In short, we are following these steps

  • Started zookeeper and verified one is leader, other is following.
  • Configured the cluster in Helix
  • Started one of the two JBoss server, which did start successfully
  • Started the second JBoss server, which failed to start (NoSuchElementException stacktrace in logs, see full stacktrace below)

We are sure both servers are configured the same way, both are starting successfully when they are the first instance to start, both do crash when starting second. Apparently, the servers can register themselves in the cluster only when they are the first one to register.

...
jboss.deployment.unit."business-central.war".WeldStartService: org.jboss.msc.service.StartException in service jboss.deployment.unit."business-central.war".WeldStartService: Failed to start service
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1936) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]
Caused by: org.jboss.weld.exceptions.DeploymentException: Exception List with 1 exceptions:
Exception 0 :
org.jboss.weld.exceptions.WeldException: WELD-000049 Unable to invoke [method] @PostConstruct public org.jbpm.services.cdi.impl.query.QueryServiceCDIImpl.init() on org.jbpm.services.cdi.impl.query.QueryServiceCDIImpl@69141082
    at org.jboss.weld.bean.AbstractClassBean.defaultPostConstruct(AbstractClassBean.java:405)
    at org.jboss.weld.bean.ManagedBean$ManagedBeanInjectionTarget.postConstruct(ManagedBean.java:178)
    at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:298)
    at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:104)
    at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:90)
    at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:79)
    at org.jbpm.services.cdi.impl.query.QueryServiceCDIImpl$Proxy$_$$_WeldClientProxy.toString(QueryServiceCDIImpl$Proxy$_$$_WeldClientProxy.java)
    at org.kie.internal.runtime.cdi.BootOnLoadExtension.runPostConstruct(BootOnLoadExtension.java:68)
    at org.kie.internal.runtime.cdi.BootOnLoadExtension.afterDeploymentValidation(BootOnLoadExtension.java:59)
    at org.kie.internal.runtime.cdi.BootOnLoadExtension$Proxy$_$$_WeldClientProxy.afterDeploymentValidation(BootOnLoadExtension$Proxy$_$$_WeldClientProxy.java)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267)
    at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52)
    at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137)
    at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263)
    at org.jboss.weld.introspector.jlr.WeldMethodImpl.invokeOnInstance(WeldMethodImpl.java:164)
    at org.jboss.weld.introspector.ForwardingWeldMethod.invokeOnInstance(ForwardingWeldMethod.java:51)
    at org.jboss.weld.injection.MethodInjectionPoint.invokeOnInstanceWithSpecialValue(MethodInjectionPoint.java:163)
    at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:245)
    at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:233)
    at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:213)
    at org.jboss.weld.bootstrap.events.AbstractContainerEvent.fire(AbstractContainerEvent.java:75)
    at org.jboss.weld.bootstrap.events.AbstractDeploymentContainerEvent.fire(AbstractDeploymentContainerEvent.java:46)
    at org.jboss.weld.bootstrap.events.AfterDeploymentValidationImpl.fire(AfterDeploymentValidationImpl.java:31)
    at org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:382)
    at org.jboss.as.weld.WeldStartService.start(WeldStartService.java:65)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.weld.util.reflection.SecureReflections$13.work(SecureReflections.java:267)
    at org.jboss.weld.util.reflection.SecureReflectionAccess.run(SecureReflectionAccess.java:52)
    at org.jboss.weld.util.reflection.SecureReflectionAccess.runAsInvocation(SecureReflectionAccess.java:137)
    at org.jboss.weld.util.reflection.SecureReflections.invoke(SecureReflections.java:263)
    at org.jboss.weld.introspector.jlr.WeldMethodImpl.invoke(WeldMethodImpl.java:168)
    at org.jboss.weld.bean.AbstractClassBean.defaultPostConstruct(AbstractClassBean.java:403)
    ... 33 more
Caused by: java.util.NoSuchElementException
    at java.util.ArrayList$Itr.next(ArrayList.java:854)
    at org.uberfire.java.nio.fs.jgit.JGitFileSystem$1$1.next(JGitFileSystem.java:194)
    at org.uberfire.java.nio.fs.jgit.JGitFileSystem$1$1.next(JGitFileSystem.java:173)
    at org.dashbuilder.dataset.DataSetDefRegistryCDI.initFileSystem(DataSetDefRegistryCDI.java:141)
    at org.dashbuilder.dataset.DataSetDefRegistryCDI.<init>(DataSetDefRegistryCDI.java:110)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
    at org.jboss.weld.introspector.jlr.WeldConstructorImpl.newInstance(WeldConstructorImpl.java:206)
    at org.jboss.weld.injection.ConstructorInjectionPoint.newInstance(ConstructorInjectionPoint.java:128)
    at org.jboss.weld.bean.ManagedBean.createInstance(ManagedBean.java:340)
    at org.jboss.weld.bean.ManagedBean$ManagedBeanInjectionTarget.produce(ManagedBean.java:204)
    at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:296)
    at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:104)
    at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:90)
    at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:79)
    at org.dashbuilder.dataset.DataSetDefRegistryCDI$Proxy$_$$_WeldClientProxy.getDataSetDef(DataSetDefRegistryCDI$Proxy$_$$_WeldClientProxy.java)
    at org.jbpm.kie.services.impl.query.QueryServiceImpl.registerQuery(QueryServiceImpl.java:122)
    at org.jbpm.kie.services.impl.query.QueryServiceImpl.init(QueryServiceImpl.java:109)
    at org.jbpm.services.cdi.impl.query.QueryServiceCDIImpl.init(QueryServiceCDIImpl.java:69)
    ... 43 more

    at org.jboss.weld.bootstrap.events.AbstractDeploymentContainerEvent.fire(AbstractDeploymentContainerEvent.java:48)
    at org.jboss.weld.bootstrap.events.AfterDeploymentValidationImpl.fire(AfterDeploymentValidationImpl.java:31)
    at org.jboss.weld.bootstrap.WeldBootstrap.validateBeans(WeldBootstrap.java:382)
    at org.jboss.as.weld.WeldStartService.start(WeldStartService.java:65)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1980) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1913) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    ... 3 more

Environment

  • Red Hat JBoss BRMS (BRMS)
    • 6.4.0
  • Red Hat JBoss BPM Suite (BPMS)
    • 6.4.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content