Getting exception while configuring Infinispan as Hibernate 2nd Level Cache in Persistence.xml in EAP6

Issue

• User have deployed EAR which contains a lib jar file with `persistence.xml and Hibernate 2LC where cli reload fails.

• User have an EAR file containing two ejb-jars and some more jars in the lib directory, and a lib/xxx-persistence-nnn.jar containing a META-INF/persistence.xml defining the JPA mappings as well as usage of Infinispan 2LC. The structure conforms to the JSR-317 Spec (JPA 2.0) Chapter 8, where it says, persistence context (META-INF/persistence.xml) can be defined in a lib jar file. User made it that way (before user had the persistence.xml placed in the ejb-jar) in order to deploy the EJB packages in more than one EAR, each defining a global persistence Ctx and one 2LC.

• Deployment and undeployment as well as deploy --force (all using cli) run without errors, but calling the cli command reload fails. User need the reload command in order to switch between admin mode (during the configuration phase, when user fire cli commands) and online mode (when user is ready with the server config).

• Curiously, the previous version containing the persistence.xml into the EJB jar package produced no error.

• User is getting below error in server.log:

14:55:38,526 ERROR [org.jboss.as.jpa] (ServerService Thread Pool -- 55) JBAS011406: Failed to stop persistence unit service test-ear-ear#test-entityManager: java.lang.IllegalStateException: Cache container has been stopped and cannot be reused. Recreate the cache container.
    at org.infinispan.manager.DefaultCacheManager.assertIsNotTerminated(DefaultCacheManager.java:936)
    at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:536)
    at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:559)
    at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:113)
    at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:104)
    at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:95)
    at org.hibernate.cache.infinispan.InfinispanRegionFactory.stopCacheRegions(InfinispanRegionFactory.java:359)
    at org.hibernate.cache.infinispan.InfinispanRegionFactory.stop(InfinispanRegionFactory.java:353)
    at org.hibernate.internal.CacheImpl.close(CacheImpl.java:289)
    at org.hibernate.internal.SessionFactoryImpl.close(SessionFactoryImpl.java:1396)
    at org.hibernate.ejb.EntityManagerFactoryImpl.close(EntityManagerFactoryImpl.java:208)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$2.run(PersistenceUnitServiceImpl.java:124)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_40]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_40]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_40]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
        ....
        ....

14:55:39,410 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 50) MSC000001: Failed to start service jboss.persistenceunit."test.ear#test-entityManager": org.jboss.msc.service.StartException in service jboss.persistenceunit."test-ear.ear#test-entityManager": javax.persistence.PersistenceException: [PersistenceUnit: test-entityManager] Unable to build EntityManagerFactory
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:103)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_40]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_40]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_40]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Final-redhat-1]
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: test-entityManager] Unable to build EntityManagerFactory
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:925)
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:900)
    at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:76)
    at... org.jboss.as.jpa.service.PersistenceUnitServiceImpl.createContainerEntityManagerFactory(PersistenceUnitServiceImpl.java:200)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl.access$600(PersistenceUnitServiceImpl.java:57)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:99)
    ... 4 more
Caused by: org.hibernate.service.spi.ServiceException: Unable to create requested service [org.hibernate.engine.spi.CacheImplementor]
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:211)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:175)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:156)
    at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:267)
    at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1799)
    at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:96)
    at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:915)
    ... 9 more
Caused by: org.hibernate.cache.CacheException: Unable to start region factory
    at org.hibernate.cache.infinispan.InfinispanRegionFactory.start(InfinispanRegionFactory.java:323)
    at org.hibernate.internal.CacheImpl.<init>(CacheImpl.java:70)
    at org.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:40)
    at org.hibernate.engine.spi.CacheInitiator.initiateService(CacheInitiator.java:35)
    at... org.hibernate.service.internal.SessionFactoryServiceRegistryImpl.initiateService(SessionFactoryServiceRegistryImpl.java:91)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:201)
    ... 15 more
Caused by: org.jboss.msc.service.DuplicateServiceException: Service jboss.hibernate."test-ear.ear#test-entityManager" is already registered
    at org.jboss.msc.service.ServiceRegistrationImpl.setInstance(ServiceRegistrationImpl.java:158) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at org.jboss.msc.service.ServiceControllerImpl.startInstallation(ServiceControllerImpl.java:236) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at org.jboss.msc.service.ServiceContainerImpl.install(ServiceContainerImpl.java:742) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at org.jboss.msc.service.ServiceBuilderImpl.install(ServiceBuilderImpl.java:345) [jboss-msc-1.1.5.Final-redhat-1.jar:1.1.5.Final-redhat-1]
    at org.jboss.as.jpa.hibernate4.infinispan.InfinispanRegionFactory.createCacheManager(InfinispanRegionFactory.java:91)
    at org.hibernate.cache.infinispan.InfinispanRegionFactory.start(InfinispanRegionFactory.java:308)
    ... 20 more

• User configured the persistence.xml as below:

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
    <persistence-unit name="test-entityManager">
        <jta-data-source>java:jboss/datasources/testDS</jta-data-source>
         <jar-file>../test.jar</jar-file>
        <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
        <properties>
            <property name="transaction.factory_class" value="org.hibernate.transaction.JTATransactionFactory" />
            <property name="hibernate.show_sql" value="true" />
            <property name="hibernate.format_sql" value="true" />
            <property name="hibernate.use_sql_comments" value="true" />
            <property name="hibernate.connection.SetBigStringTryClob" value="true" />
            <property name="hibernate.cache.use_second_level_cache" value="true" />
            <property name="hibernate.cache.use_query_cache" value="true" />
            <property name="hibernate.cache.use_structured_entries" value="true" />
            <property name="hibernate.cache.region.factory_class" value="org.jboss.as.jpa.hibernate4.infinispan.InfinispanRegionFactory"/>
            <property name="hibernate.cache.infinispan.cachemanager" value="java:jboss/infinispan/container/hibernate"/>
            <property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.JBossTransactionManagerLookup"/>
            <property name="hibernate.generate_statistics" value="true" />
        </properties>
    </persistence-unit>
</persistence>