DG 8.3.1 query deadlock in index writer threads under heavy load
Issue
Under heavy load DG 8.3.1 has might incur in a deadlock in index writer threads:
"Hibernate Search - default backend - Worker thread - 0" #98 prio=0 tid=0x62 nid=NA waiting
java.lang.Thread.State: WAITING
at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
- parking to wait for <0x603e22ea> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
at java.base@11.0.16/java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:367)
at org.hibernate.search.engine.backend.orchestration.spi.BatchingExecutor.submit(BatchingExecutor.java:116)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSerialWorkOrchestratorImpl.doSubmit(LuceneSerialWorkOrchestratorImpl.java:94)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSerialWorkOrchestratorImpl.doSubmit(LuceneSerialWorkOrchestratorImpl.java:21)
at org.hibernate.search.engine.backend.orchestration.spi.AbstractWorkOrchestrator.submit(AbstractWorkOrchestrator.java:135)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSerialWorkOrchestratorImpl.submit(LuceneSerialWorkOrchestratorImpl.java:21)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSerialWorkOrchestrator.submit(LuceneSerialWorkOrchestrator.java:24)
at org.hibernate.search.backend.lucene.work.execution.impl.LuceneIndexIndexer.submit(LuceneIndexIndexer.java:125)
at org.hibernate.search.backend.lucene.work.execution.impl.LuceneIndexIndexer.addOrUpdate(LuceneIndexIndexer.java:70)
at org.hibernate.search.mapper.pojo.work.impl.PojoTypeIndexer.addOrUpdate(PojoTypeIndexer.java:82)
at org.hibernate.search.mapper.pojo.work.impl.PojoIndexerImpl.addOrUpdate(PojoIndexerImpl.java:48)
at org.infinispan.search.mapper.work.impl.SearchIndexerImpl.addOrUpdate(SearchIndexerImpl.java:48)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:378)
at org.infinispan.query.backend.QueryInterceptor.lambda$processChange$7(QueryInterceptor.java:428)
at org.infinispan.query.backend.QueryInterceptor$$Lambda$1430/0x0000000840cc9c40.apply(Unknown Source)
at java.base@11.0.16/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
at java.base@11.0.16/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base@11.0.16/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneBatchedWork.submitTo(LuceneBatchedWork.java:27)
at org.hibernate.search.backend.lucene.orchestration.impl.LuceneBatchedWork.submitTo(LuceneBatchedWork.java:14)
at org.hibernate.search.engine.backend.orchestration.spi.BatchingExecutor$BatchWorker.work(BatchingExecutor.java:173)
at org.hibernate.search.engine.backend.orchestration.spi.SingletonTask$RunnableWrapper.run(SingletonTask.java:191)
at java.base@11.0.16/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base@11.0.16/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
Locked ownable synchronizers:
- <0x249111b9> (a java.util.concurrent.ThreadPoolExecutor$Worker)
Environment
- Red Hat Data Grid (RHDG)
- 8.x
- Hibernate Search
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.