Chapter 6. Data Virtualization

Data Virtualization is available as a Technology Preview feature in Red Hat Integration 2020-Q2. Data Virtualization is a container-native service, based on the Teiid open source project, that provides integrated access to a range of data sources, including relational databases, MongoDB, and Salesforce, through a single uniform API.

Applications and users connect to a virtual database over standard interfaces (OData REST, or JDBC/ODBC) and can use SQL to interact with the integrated data in the same way that they would interact with a single relational database.

Important

Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend implementing any Technology Preview features in production environments.

This Technology Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about support scope, see Technology Preview Features Support Scope.

6.1. Data Virtualization enhancements

Red Hat Integration 2020-Q2 provides the following enhancements for Data Virtualization:

  • Simplified format for defining data sources in the custom resource for a virtual database.

  • Compatibility with the following additional data sources:

    • Ceph
    • SFTP
    • SOAP
  • Ability to import from existing virtual databases.
  • Custom Prometheus metrics.
  • Ability to configure a load balancer for JDBC/ODBC traffic.
  • Jaeger integration.
  • Improved handling of TLS certificates for encrypting connections from clients and to data sources.
  • Ability to configure a custom Maven settings file.
  • Passing down proxy settings to operands.

  • New user documentation:

6.2. Data Virtualization known issues

ENTESB-14311 - Data Grid based materialization does not work

Materialized views that are stored in the Data Grid cache can not be configured in this release. When a virtual database is deployed into a namespace where the Data Grid or Infinispan Operators are present, and a virtual database has configured a view with materialization (for example, OPTIONS (MATERIALIZED 'TRUE')), the Data Virtualization Operator automatically creates a Data Grid cache store, for the purposes of materialization. However, after the virtual database is deployed, queries that you submit to fetch the contents of the view return the following error: 

TEIID30384 Error while evaluating function mvstatus

To resolve the error, disable use of the Data Grid or Infinispan cache, and revert to using the default materialization mechanism by completing the following steps.

  1. Set the environment property DISABLE_ISPN_CACHING in the CR to true. For example, 

    apiVersion: teiid.io/v1alpha1
kind: VirtualDatabase
metadata:
  name: matexample
spec:
  replicas: 1
  env:
    - name: DISABLE_ISPN_CACHING
      value: "true"
  build:
    source:
 ....

  2. Run the Data Virtualization Operator with the updated CR.

    After you build the virtual database with the specified environment variable, the caching mechanism reverts to the default method.

ENTESB-14297 - Connecting to SAP HANA does not work with the prescribed driver

When you create a virtual database that uses SAP HANA as a data source, the JDBC driver that is specified as a build dependency for the virtual database is unable to connect to HANA. Tested using SAP HANA version 1.00.102.

ENTESB-13462 - FORMAT functions for different versions of the Data Virtualization Operator return different results

When you use format functions such as FORMATBIGDECIMAL or FORMATDOUBLE to query a virtual database, the results that the functions return for negative numbers might not be formatted as expected. The format of the result depends on the underlying Java version specified by the Data Virtualization Operator. Because recent versions of the Operator switched from using Java 8 to Java 11, you might experience changes in the formatting of your query results.

In Java versions earlier than Java 11, regardless of whether you specify a minus sign character (-) as a prefix or suffix of a negative format pattern, the returned negative result string is preceded by a minus sign. However, in Java 11, if you append a minus sign to a negative format pattern, the result string that is returned has the minus sign appended to it.

If you obtain unexpected results for negative numbers that a parsing or formatting function returns, revise the pattern string for the function so that the minus sign precedes the rest of the pattern.

Example

The following query specifies the negative format pattern, #,0;#0-. 

SELECT FORMATBIGDECIMAL((99 - 10000000), '#,##0;###0-')

Depending on the version of the Operator, one of the following results is returned:

DV Operators using Java 8
-9,999,901
DV Operators using Java 11

9,999,901-

To achieve the same result as with Java 8, modify the formatting pattern for the query to ,#0;-#.

6.3. Data Virtualization migration considerations

You can migrate virtual database that you developed for JBoss Data Virtualization or Teiid deployments. For information about migrating legacy virtual databases to Data Virtualization, see Using Data Virtualization.