Red Hat Training

A Red Hat training course is available for Red Hat JBoss Data Virtualization

Chapter 11. Built-in Text Extractors

The hierarchical database comes with a single text extractor. All you have to do is configure it and be ready to work with the generated output.

11.1. Tika Text Extractor

This text extractor uses the Tika library to extract text from a variety of file formats. It will automatically discover all of the Tika Parser implementations that are defined in META-INF/services/org.apache.tika.parser.Parser text files accessible via the current classloader and that contain the class names of the Parser implementations (one class name per line in each file). In other words, ensure that the Tika libraries for the appropriate file formats are on the classpath, and the text extractor will be able to use them all.
This text extractor can be configured in a hierarchical database configuration by specifying several optional properties:
  • excludedMimeTypes - The comma- or whitespace-separated list of MIME types that should be excluded from text extraction, even if there is a Tika Parser available for that MIME type. By default, the MIME types for package files are excluded, though explicitly setting any excluded MIME types will override these default.
  • includedMimeTypes - The comma- or whitespace-separated list of MIME types that should be included in text extraction. This extractor will ignore any MIME types in this list that are not covered by Tika Parser implementations.
To use this extractor, include the modeshape-extractor-tika JAR and the appropriate required Tika JARs are on the classpath (or via Maven) and configure the repository in a similar fashion to: 
{
    "name" : "Sample Config",
    "query" : {
        "textExtracting": {
            "extractors" : {
                "tikaExtractor":{
                    "name" : "General content-based extractor",
                    "classname" : "tika",
                }
            }
         },
    }
}