Automated Hibernate Search index generation is slower than manual generation in EAP 6
Issue
- Index generation is signficantly slower (2 minutes) for automated index generation compared to
hibernate.search.indexing_strategy=manual(2 seconds) - Using
directory_provider=filesystem-master(to support clustering between two nodes)
"http-/192.168.1.2:8080-5", for thread dumps 5 - 10 (17:30:45 => 17:31:40) is executing the following:
"http-/192.168.1.2:8080-5" #202 daemon prio=5 os_prio=0 tid=0x000000001ea4a000 nid=0xe78 waiting on condition [0x0000000035bfb000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor.parkCurrentThread(SyncWorkProcessor.java:172)
at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor.submit(SyncWorkProcessor.java:104)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyWork(LuceneBackendQueueProcessor.java:101)
at org.hibernate.search.indexes.impl.DirectoryBasedIndexManager.performOperations(DirectoryBasedIndexManager.java:126)
at org.hibernate.search.backend.impl.WorkQueuePerIndexSplitter.commitOperations(WorkQueuePerIndexSplitter.java:63)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.performWorks(BatchedQueueingProcessor.java:99)
at org.hibernate.search.backend.impl.PostTransactionWorkQueueSynchronization.afterCompletion(PostTransactionWorkQueueSynchronization.java:105)
at org.hibernate.search.backend.impl.EventSourceTransactionContext$DelegateToSynchronizationOnAfterTx.doAfterTransactionCompletion(EventSourceTransactionContext.java:209)
at org.hibernate.engine.spi.ActionQueue$AfterTransactionCompletionProcessQueue.afterTransactionCompletion(ActionQueue.java:792)
...
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1189)
...
at com.myco.Employee.saveEmployee(Employee.java:999)
...
And for this same thread dump set/time period, the following stack remains unchanged (think this is what is blocking completion of the above)
"Hibernate Search sync consumer thread for index <index_name_here>" #143 daemon prio=5 os_prio=0 tid=0x0000000026e1d000 nid=0xb40 runnable [0x000000002f92e000]
java.lang.Thread.State: RUNNABLE
at java.io.RandomAccessFile.open0(Native Method)
at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:71)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:98)
at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:58)
at org.apache.lucene.index.SegmentReader.openNorms(SegmentReader.java:639)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:121)
at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:696)
- locked <0x000000078cdf03e0> (a org.apache.lucene.index.IndexWriter$ReaderPool)
at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:671)
- locked <0x000000078cdf03e0> (a org.apache.lucene.index.IndexWriter$ReaderPool)
at org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:244)
- locked <0x000000078cdfbe80> (a org.apache.lucene.index.BufferedDeletesStream)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3615)
- locked <0x000000078cdeb2a8> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3376)
- locked <0x000000078cdeb2a8> (a org.apache.lucene.index.IndexWriter)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3485)
- locked <0x000000078cdf0428> (a java.lang.Object)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3467)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3451)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.commitIndexWriter(IndexWriterHolder.java:158)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.commitIndexWriter(IndexWriterHolder.java:171)
at org.hibernate.search.backend.impl.lucene.PerChangeSetCommitPolicy.onChangeSetApplied(PerChangeSetCommitPolicy.java:46)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.afterTransactionApplied(AbstractWorkspaceImpl.java:107)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:124)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67)
at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:162)
at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:152)
at java.lang.Thread.run(Thread.java:745)
Environment
- Red Hat JBoss Enterprise Application Platform (EAP) 6.4
- Hibernate Search
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.