Chapter 7. Querying
Hibernate Search can execute Lucene queries and retrieve domain objects managed by an Hibernate session. The search provides the power of Lucene without leaving the Hibernate paradigm, giving another dimension to the Hibernate classic search mechanisms (HQL, Criteria query, native SQL query).
Preparing and executing a query consists of following four steps:
- Creating a
FullTextSession - Creating a Lucene query using either Hibernate Search query DSL (recommended) or using the Lucene Query API
- Wrapping the Lucene query using an
org.hibernate.Query - Executing the search by calling for example
list()orscroll()
To access the querying facilities, use a
FullTextSession. This Search specific session wraps a regular org.hibernate.Session in order to provide query and indexing capabilities.
Example 7.1. Creating a FullTextSession
Session session = sessionFactory.openSession(); ... FullTextSession fullTextSession = Search.getFullTextSession(session);
Use the
FullTextSession to build a full-text query using either the Hibernate Search query DSL or the native Lucene query.
Use the following code when using the Hibernate Search query DSL:
final QueryBuilder b = fullTextSession.getSearchFactory().buildQueryBuilder().forEntity( Myth.class ).get(); org.apache.lucene.search.Query luceneQuery = b.keyword() .onField("history").boostedTo(3) .matching("storm") .createQuery(); org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery ); List result = fullTextQuery.list(); //return a list of managed objects
As an alternative, write the Lucene query using either the Lucene query parser or the Lucene programmatic API.
Example 7.2. Creating a Lucene query via the QueryParser
SearchFactory searchFactory = fullTextSession.getSearchFactory(); org.apache.lucene.queryParser.QueryParser parser = new QueryParser("title", searchFactory.getAnalyzer(Myth.class) ); try { org.apache.lucene.search.Query luceneQuery = parser.parse( "history:storm^3" ); } catch (ParseException e) { //handle parsing failure } org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery(luceneQuery); List result = fullTextQuery.list(); //return a list of managed objects
A Hibernate query built on the Lucene query is a
org.hibernate.Query. This query remains in the same paradigm as other Hibernate query facilities, such as HQL (Hibernate Query Language), Native, and Criteria. Use methods such as list(), uniqueResult(), iterate() and scroll() with the query.
The same extensions are available with the Hibernate Java Persistence APIs:
Example 7.3. Creating a Search query using the JPA API
EntityManager em = entityManagerFactory.createEntityManager(); FullTextEntityManager fullTextEntityManager = org.hibernate.search.jpa.Search.getFullTextEntityManager(em); ... final QueryBuilder b = fullTextEntityManager.getSearchFactory() .buildQueryBuilder().forEntity( Myth.class ).get(); org.apache.lucene.search.Query luceneQuery = b.keyword() .onField("history").boostedTo(3) .matching("storm") .createQuery(); javax.persistence.Query fullTextQuery = fullTextEntityManager.createFullTextQuery( luceneQuery ); List result = fullTextQuery.getResultList(); //return a list of managed objects
Note
The following examples we will use the Hibernate APIs but the same example can be easily rewritten with the Java Persistence API by just adjusting the way the
FullTextQuery is retrieved.
7.1. Building Queries
Hibernate Search queries are built on Lucene queries, allowing users to use any Lucene query type. When the query is built, Hibernate Search uses org.hibernate.Query as the query manipulation API for further query processing.
7.1.1. Building a Lucene Query Using the Lucene API
With the Lucene API, use either the query parser (simple queries) or the Lucene programmatic API (complex queries). Building a Lucene query is out of scope for the Hibernate Search documentation. For details, see the online Lucene documentation or a copy of Lucene in Action or Hibernate Search in Action.
7.1.2. Building a Lucene Query
Using the Lucene programmatic API, it is possible to write full-text queries. However, when using Lucene programmatic API, the parameters must be converted to their string equivalent and must also apply the correct analyzer to the right field. A ngram analyzer for example uses several ngrams as the tokens for a given word and should be searched as such. It is recommended to use the
QueryBuilder for this task.
The Hibernate Search query API is fluent. This API has a following key characteristics:
- Method names are in English. As a result, API operations can be read and understood as a series of English phrases and instructions.
- It uses IDE autocompletion which helps possible completions for the current input prefix and allows the user to choose the right option.
- It often uses the chaining method pattern.
- It is easy to use and read the API operations.
To use the API, first create a query builder that is attached to a given indexedentitytype. This
QueryBuilder knows what analyzer to use and what field bridge to apply. Several QueryBuilders (one for each entity type involved in the root of your query) can be created. The QueryBuilder is derived from the SearchFactory.
QueryBuilder mythQB = searchFactory.buildQueryBuilder().forEntity( Myth.class ).get();
The analyzer, used for a given field or fields can also be overridden.
QueryBuilder mythQB = searchFactory.buildQueryBuilder() .forEntity( Myth.class ) .overridesForField("history","stem_analyzer_definition") .get();
The query builder is now used to build Lucene queries. Customize queries generated using Lucene's query parser or
Query objects assembled using the Lucene programmatic API are used with the Hibernate Search DSL.
7.1.3. Keyword Queries
The following example shows how to search for a specific word:
Query luceneQuery = mythQB.keyword().onField("history").matching("storm").createQuery();
Table 7.1. Keyword query parameters
| Parameter | Description |
|---|---|
| keyword() | Use this parameter to find a specific word |
| onField() | Use this parameter to specify in which lucene field to search the word |
| matching() | use this parameter to specify the match for search string |
| createQuery() | creates the Lucene query object |
- The value "storm" is passed through the
historyFieldBridge. This is useful when numbers or dates are involved. - The field bridge value is then passed to the analyzer used to index the field
history. This ensures that the query uses the same term transformation than the indexing (lower case, ngram, stemming and so on). If the analyzing process generates several terms for a given word, a boolean query is used with theSHOULDlogic (roughly anORlogic).
To search a property that is not of type string.
@Indexed public class Myth { @Field(analyze = Analyze.NO) @DateBridge(resolution = Resolution.YEAR) public Date getCreationDate() { return creationDate; } public Date setCreationDate(Date creationDate) { this.creationDate = creationDate; } private Date creationDate; ... } Date birthdate = ...; Query luceneQuery = mythQb.keyword().onField("creationDate").matching(birthdate).createQuery();
Note
In plain Lucene, the
Date object had to be converted to its string representation (in this case the year)
This conversion works for any object, provided that the
FieldBridge has an objectToString method (and all built-in FieldBridge implementations do).
The next example searches a field that uses ngram analyzers. The ngram analyzers index succession of ngrams of words, which helps to avoid user typos. For example, the 3-grams of the word hibernate are hib, ibe, ber, rna, nat, ate.
@AnalyzerDef(name = "ngram", tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class ), filters = { @TokenFilterDef(factory = StandardFilterFactory.class), @TokenFilterDef(factory = LowerCaseFilterFactory.class), @TokenFilterDef(factory = StopFilterFactory.class), @TokenFilterDef(factory = NGramFilterFactory.class, params = { @Parameter(name = "minGramSize", value = "3"), @Parameter(name = "maxGramSize", value = "3") } ) } ) public class Myth { @Field(analyzer=@Analyzer(definition="ngram") @DateBridge(resolution = Resolution.YEAR) public String getName() { return name; } public String setName(Date name) { this.name = name; } private String name; ... } Date birthdate = ...; Query luceneQuery = mythQb.keyword().onField("name").matching("Sisiphus") .createQuery();
The matching word "Sisiphus" will be lower-cased and then split into 3-grams: sis, isi, sip, phu, hus. Each of these ngram will be part of the query. The user is then able to find the Sysiphus myth (with a
y). All that is transparently done for the user.
Note
If the user does not want a specific field to use the field bridge or the analyzer then the
ignoreAnalyzer() or ignoreFieldBridge() functions can be called.
To search for multiple possible words in the same field, add them all in the matching clause.
//search document with storm or lightning in their history Query luceneQuery = mythQB.keyword().onField("history").matching("storm lightning").createQuery();
To search the same word on multiple fields, use the
onFields method.
Query luceneQuery = mythQB .keyword() .onFields("history","description","name") .matching("storm") .createQuery();
Sometimes, one field should be treated differently from another field even if searching the same term, use the
andField() method for that.
Query luceneQuery = mythQB.keyword() .onField("history") .andField("name") .boostedTo(5) .andField("description") .matching("storm") .createQuery();
In the previous example, only field name is boosted to 5.
7.1.4. Fuzzy Queries
To execute a fuzzy query (based on the Levenshtein distance algorithm), start like a
keyword query and add the fuzzy flag.
Query luceneQuery = mythQB .keyword() .fuzzy() .withThreshold( .8f ) .withPrefixLength( 1 ) .onField("history") .matching("starm") .createQuery();
The
threshold is the limit above which two terms are considering matching. It is a decimal between 0 and 1 and the default value is 0.5. The prefixLength is the length of the prefix ignored by the "fuzzyness". While the default value is 0, a non zero value is recommended for indexes containing a huge amount of distinct terms.
7.1.5. Wildcard Queries
Wildcard queries can also be executed (queries where some of parts of the word are unknown). The
? represents a single character and * represents any character sequence. Note that for performance purposes, it is recommended that the query does not start with either ? or *.
Query luceneQuery = mythQB .keyword() .wildcard() .onField("history") .matching("sto*") .createQuery();
Note
Wildcard queries do not apply the analyzer on the matching terms. Otherwise the risk of
* or ? being mangled is too high.
7.1.6. Phrase Queries
So far we have been looking for words or sets of words, the user can also search exact or approximate sentences. Use
phrase() to do so.
Query luceneQuery = mythQB .phrase() .onField("history") .sentence("Thou shalt not kill") .createQuery();
Approximate sentences can be searched by adding a slop factor. The slop factor represents the number of other words permitted in the sentence: this works like a within or near operator.
Query luceneQuery = mythQB .phrase() .withSlop(3) .onField("history") .sentence("Thou kill") .createQuery();
7.1.7. Range Queries
A range query searches for a value in between given boundaries (included or not) or for a value below or above a given boundary (included or not).
//look for 0 <= starred < 3 Query luceneQuery = mythQB .range() .onField("starred") .from(0).to(3).excludeLimit() .createQuery(); //look for myths strictly BC Date beforeChrist = ...; Query luceneQuery = mythQB .range() .onField("creationDate") .below(beforeChrist).excludeLimit() .createQuery();
7.1.8. Combining Queries
Queries can be aggregated (combine) to create more complex queries. The following aggregation operators are available:
SHOULD: the query should contain the matching elements of the subquery.MUST: the query must contain the matching elements of the subquery.MUST NOT: the query must not contain the matching elements of the subquery.
The subqueries can be any Lucene query including a boolean query itself. Following are some examples:
//look for popular modern myths that are not urban Date twentiethCentury = ...; Query luceneQuery = mythQB .bool() .must( mythQB.keyword().onField("description").matching("urban").createQuery() ) .not() .must( mythQB.range().onField("starred").above(4).createQuery() ) .must( mythQB .range() .onField("creationDate") .above(twentiethCentury) .createQuery() ) .createQuery(); //look for popular myths that are preferably urban Query luceneQuery = mythQB .bool() .should( mythQB.keyword().onField("description").matching("urban").createQuery() ) .must( mythQB.range().onField("starred").above(4).createQuery() ) .createQuery(); //look for all myths except religious ones Query luceneQuery = mythQB .all() .except( monthQb .keyword() .onField( "description_stem" ) .matching( "religion" ) .createQuery() ) .createQuery();
7.1.9. Query Options
The following is a summary of query options for query types and fields:
boostedTo(on query type and on field) boosts the whole query or the specific field to a given factorwithConstantScore(on query) returns all results that match the query have a constant score equals to the boostfilteredBy(Filter)(on query) filters query results using theFilterinstanceignoreAnalyzer(on field) ignores the analyzer when processing this fieldignoreFieldBridge(on field) ignores field bridge when processing this field
The following example illustrates how to use these options:
Query luceneQuery = mythQB .bool() .should( mythQB.keyword().onField("description").matching("urban").createQuery() ) .should( mythQB .keyword() .onField("name") .boostedTo(3) .ignoreAnalyzer() .matching("urban").createQuery() ) .must( mythQB .range() .boostedTo(5).withConstantScore() .onField("starred").above(4).createQuery() ) .createQuery();
As you can see, the Hibernate Search query DSL is an easy to use and easy to read query API and by accepting and producing Lucene queries, you can easily incorporate query types not (yet) supported by the DSL. Please give us feedback!
7.1.10. Build a Hibernate Search Query
7.1.10.1. Generality
After building the Lucene query, wrap it within a Hibernate query. The query searches all indexed entities and returns all types of indexed classes unless explicitly configured not to do so.
Example 7.4. Wrapping a Lucene Query in a Hibernate Query
FullTextSession fullTextSession = Search.getFullTextSession( session ); org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery );
For improved performance, restrict the returned types as follows:
Example 7.5. Filtering the Search Result by Entity Type
fullTextQuery = fullTextSession .createFullTextQuery( luceneQuery, Customer.class ); // or fullTextQuery = fullTextSession .createFullTextQuery( luceneQuery, Item.class, Actor.class );
The first part of the second example only returns the matching
Customers. The second part of the same example returns matching Actors and Items. The type restriction is polymorphic. As a result, if the two subclasses Salesman and Customer of the base class Person return, specify Person.class to filter based on result types.
7.1.10.2. Pagination
To avoid performance degradation, it is recommended to restrict the number of returned objects per query. A user navigating from one page to another page is a very common use case. The way to define pagination is similar to defining pagination in a plain HQL or Criteria query.
Example 7.6. Defining pagination for a search query
org.hibernate.Query fullTextQuery = fullTextSession.createFullTextQuery( luceneQuery, Customer.class ); fullTextQuery.setFirstResult(15); //start from the 15th element fullTextQuery.setMaxResults(10); //return 10 elements
Note
It is still possible to get the total number of matching elements regardless of the pagination via
fulltextQuery.getResultSize()
7.1.10.3. Sorting
Apache Lucene contains a flexible and powerful result sorting mechanism. The default sorting is by relevance and is appropriate for a large variety of use cases. The sorting mechanism can be changed to sort by other properties using the Lucene Sort object to apply a Lucene sorting strategy.
Example 7.7. Specifying a Lucene Sort
org.hibernate.search.FullTextQuery query = s.createFullTextQuery( query, Book.class ); org.apache.lucene.search.Sort sort = new Sort( new SortField("title", SortField.STRING)); query.setSort(sort); List results = query.list();
Note
Fields used for sorting must not be tokenized. For more information about tokenizing, see Section 6.1.1.2, “@Field”.
7.1.10.4. Fetching Strategy
Hibernate Search loads objects using a single query if the return types are restricted to one class. Hibernate Search is restricted by the static fetching strategy defined in the domain model. It is useful to refine the fetching strategy for a specific use case as follows:
Example 7.8. Specifying FetchMode on a query
Criteria criteria = s.createCriteria( Book.class ).setFetchMode( "authors", FetchMode.JOIN ); s.createFullTextQuery( luceneQuery ).setCriteriaQuery( criteria );
In this example, the query will return all Books matching the LuceneQuery. The authors collection will be loaded from the same query using an SQL outer join.
In a criteria query definition, the type is guessed based on the provided criteria query. As a result, it is not necessary to restrict the return entity types.
Important
The fetch mode is the only adjustable property. Do not use a restriction (a where clause) on the
Criteria query because the getResultSize() throws a SearchException if used in conjunction with a Criteria with restriction.
If more than one entity is expected, do not use
setCriteriaQuery.
7.1.10.5. Projection
In some cases, only a small subset of the properties is required. Use Hibernate Search to return a subset of properties as follows:
Hibernate Search extracts properties from the Lucene index and converts them to their object representation and returns a list of
Object[]. Projections prevent a time consuming database round-trip. However, they have following constraints:
- The properties projected must be stored in the index (
@Field(store=Store.YES)), which increases the index size. - the properties projected must use a
FieldBridgeimplementingorg.hibernate.search.bridge.TwoWayFieldBridgeororg.hibernate.search.bridge.TwoWayStringBridge, the latter being the simpler version.Note
All Hibernate Search built-in types are two-way. - Only the simple properties of the indexed entity or its embedded associations can be projected. Therefore a whole embedded entity cannot be projected.
- Projection does not work on collections or maps which are indexed via
@IndexedEmbedded
Lucene provides metadata information about query results. Use projection constants to retrieve the metadata.
Example 7.9. Using Projection to Retrieve Metadata
org.hibernate.search.FullTextQuery query = s.createFullTextQuery( luceneQuery, Book.class ); query.setProjection( FullTextQuery.SCORE, FullTextQuery.THIS, "mainAuthor.name" ); List results = query.list(); Object[] firstResult = (Object[]) results.get(0); float score = firstResult[0]; Book book = firstResult[1]; String authorName = firstResult[2];
Fields can be mixed with the following projection constants:
FullTextQuery.THIS: returns the initialized and managed entity (as a non projected query would have done).FullTextQuery.DOCUMENT: returns the Lucene Document related to the object projected.FullTextQuery.OBJECT_CLASS: returns the class of the indexed entity.FullTextQuery.SCORE: returns the document score in the query. Scores are handy to compare one result against an other for a given query but are useless when comparing the result of different queries.FullTextQuery.ID: the id property value of the projected object.FullTextQuery.DOCUMENT_ID: the Lucene document id. Careful, Lucene document id can change overtime between two different IndexReader opening.FullTextQuery.EXPLANATION: returns the Lucene Explanation object for the matching object/document in the given query. This is not suitable for retrieving large amounts of data. Running explanation typically is as costly as running the whole Lucene query per matching element. As a result, projection is recommended.
7.1.10.6. Customizing Object Initialization Strategies
By default, Hibernate Search uses the most appropriate strategy to initialize entities matching the full text query. It executes one (or several) queries to retrieve the required entities. This approach minimizes database trips where few of the retrieved entities are present in the persistence context (the session) or the second level cache.
If entities are present in the second level cache, force Hibernate Search to look into the cache before retrieving a database object.
Example 7.10. Check the second-level cache before using a query
FullTextQuery query = session.createFullTextQuery(luceneQuery, User.class); query.initializeObjectWith( ObjectLookupMethod.SECOND_LEVEL_CACHE, DatabaseRetrievalMethod.QUERY );
ObjectLookupMethod defines the strategy to check if an object is easily accessible (without fetching it from the database). Other options are:
ObjectLookupMethod.PERSISTENCE_CONTEXTis used if many matching entities are already loaded into the persistence context (loaded in theSessionorEntityManager).ObjectLookupMethod.SECOND_LEVEL_CACHEchecks the persistence context and then the second-level cache.
Set the following to search in the second-level cache:
- Correctly configure and activate the second-level cache.
- Enable the second-level cache for the relevant entity. This is done using annotations such as
@Cacheable. - Enable second-level cache read access for either
Session,EntityManagerorQuery. UseCacheMode.NORMALin Hibernate native APIs orCacheRetrieveMode.USEin Java Persistence APIs).
Warning
Unless the second-level cache implementation is EHCache or Infinispan, do not use
ObjectLookupMethod.SECOND_LEVEL_CACHE. Other second-level cache providers do not implement this operation efficiently.
Customize how objects are loaded from the database using
DatabaseRetrievalMethod as follows:
QUERY(default) uses a set of queries to load several objects in each batch. This approach is recommended.FIND_BY_IDloads one object at a time using theSession.getorEntityManager.findsemantic. This is recommended if the batch size is set for the entity, which allows Hibernate Core to load entities in batches.
7.1.10.7. Limiting the Time of a Query
Limit the time a query takes in Hibernate Guide as follows:
- Raise an exception when arriving at the limit.
- Limit to the number of results retrieved when the time limit is raised.
7.1.10.8. Raise an Exception on Time Limit
If a query uses more than the defined amount of time, a
QueryTimeoutException is raised (org.hibernate.QueryTimeoutException or javax.persistence.QueryTimeoutException depending on the programmatic API).
To define the limit when using the native Hibernate APIs, use one of the following approaches:
Example 7.11. Defining a Timeout in Query Execution
Query luceneQuery = ...; FullTextQuery query = fullTextSession.createFullTextQuery(luceneQuery, User.class); //define the timeout in seconds query.setTimeout(5); //alternatively, define the timeout in any given time unit query.setTimeout(450, TimeUnit.MILLISECONDS); try { query.list(); } catch (org.hibernate.QueryTimeoutException e) { //do something, too slow }
The
getResultSize(), iterate() and scroll() honor the timeout until the end of the method call. As a result, Iterable or the ScrollableResults ignore the timeout. Additionally, explain() does not honor this timeout period. This method is used for debugging and to check the reasons for slow performance of a query.
The following is the standard way to limit execution time using the Java Persistence API (JPA):
Example 7.12. Defining a Timeout in Query Execution
Query luceneQuery = ...; FullTextQuery query = fullTextEM.createFullTextQuery(luceneQuery, User.class); //define the timeout in milliseconds query.setHint( "javax.persistence.query.timeout", 450 ); try { query.getResultList(); } catch (javax.persistence.QueryTimeoutException e) { //do something, too slow }
Important
The example code does not guarantee that the query stops at the specified results amount.