Chapter 4. Creating a virtual database as a Maven artifact

You can use a Teiid Maven plugin to convert a DDL file into a Maven artifact. You define the structure of the virtual database in a DDL file and use the file to generate an artifact to deploy to a Maven repository. The Data Virtualization Operator can then deploy the artifact from the Maven repository to an OpenShift project.

This is an advanced method that provides a high level of flexibility and is suitable for complex projects. Using this method, you can create multi-module Maven projects in which you import one or more other virtual databases and incorporate them into your design.

You specify use of the Teiid plugin in your pom.xml file. You can also define other Maven dependencies in the pom.xml file. When you run the build, the plugin reads the file and resolves its contents.

Advantages of creating a virtual database as a Maven artifact
  • Flexible, clean separation between the DDL code that represents the virtual database and other configuration settings.
  • Enables easy deployment into multiple environments.
  • Provides for versioning at the virtual database level.
  • Enables a virtual database to be shared across projects and teams in a consistent way.
  • Supports continuous integration and continuous delivery (CI/CD) workflows.
Disadvantages of creating a virtual database as a Maven artifact
  • Requires a working knowledge of Maven.

Prerequisites

  • You have a compatible data source and the OpenShift cluster can access it.
  • You know how to create a pom.xml file to specify the dependencies that are required to build your virtual database.
  • You have information about the connection settings for your data sources, including login credentials.
  • The Data Virtualization Operator has access to the Maven repositories that contain build dependencies for the virtual database.
  • You have Maven 3.2 or later installed.

Procedure

  1. From a text editor, create a POM file to define build dependencies. For example,

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
      <groupId>org.teiid</groupId>
      <artifactId>dv-customer</artifactId>
      <name>dv-customer</name>
      <description>Demo project to showcase maven based vdb</description>
      <packaging>vdb</packaging>
      <version>1.0</version>
    
      <build>
        <plugins>
          <plugin>
            <groupId>org.teiid</groupId>
            <artifactId>vdb-plugin</artifactId>
            <version>1.2.0</version>
            <extensions>true</extensions>
            <executions>
              <execution>
                <goals>
                  <goal>vdb</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </project>
  2. Create a Maven project to import the virtual database definition from a DDL file. For example:

    vdb-project
    ├── pom.xml
    └── src
        └── main
            └── vdb
                └── vdb.ddl
  3. If you do not already have one, create a DDL file to specify the structure of the virtual database, and save it with a .ddl extension to the /src/main/vdb directory of your project. For example vdb.ddl

    The following example shows a sample DDL file for a virtual database that uses a postgreSQL data source:

    Example: vdb.ddl

    CREATE DATABASE customer OPTIONS (ANNOTATION 'Customer VDB');
    USE DATABASE customer;
    
    CREATE FOREIGN DATA WRAPPER postgresql;
    CREATE SERVER sampledb TYPE 'NONE' FOREIGN DATA WRAPPER postgresql;
    
    CREATE SCHEMA accounts SERVER sampledb;
    CREATE VIRTUAL SCHEMA portfolio;
    
    SET SCHEMA accounts;
    IMPORT FOREIGN SCHEMA public FROM SERVER sampledb INTO accounts OPTIONS("importer.useFullSchemaName" 'false');
    
    SET SCHEMA portfolio;
    
    CREATE VIEW CustomerZip(id bigint PRIMARY KEY, name string, ssn string, zip string) AS
       SELECT c.ID as id, c.NAME as name, c.SSN as ssn, a.ZIP as zip
       FROM accounts.CUSTOMER c LEFT OUTER JOIN accounts.ADDRESS a
       ON c.ID = a.CUSTOMER_ID;

    For information about how to use DDL to define a virtual database, see DDL metadata for schema objects in the Data Virtualization Reference. Defining the complete DDL is beyond the scope of this document.

  4. Build the virtual database artifact. Open a terminal window to the root folder of your Maven project, and type the following command:

    mvn clean install

    The command generates a ${project.name}-$2020.Q1.vdb file in your target repository.

  5. Deploy the artifact to a remote repository by typing the following command:

    mvn clean install deploy

After the virtual database artifact is available in a Maven repository, you can use a YAML-based custom resource to deploy the virtual database to OpenShift. For information about using YAML to create a custom resource for deploying virtual database Maven artifacts, see Section 4.1, “Creating a custom resource (CR) to deploy a Maven artifact”.

For information about using the Data Virtualization Operator to deploy a virtual database, see Chapter 6, Running the data virtualization operator to deploy a virtual database.

4.1. Creating a custom resource (CR) to deploy a Maven artifact

Before you can deploy a virtualization that you create as a Maven artifact, you must create a CR that defines the location of the Maven repository. When you are ready to deploy the virtualization, you provide this CR to the Data Virtualization Operator.

Prerequisites

  • You created a virtualization according to the instructions in Chapter 4, Creating a virtual database as a Maven artifact.
  • You deployed the virtualization to a Maven repository that the Data Virtualization Operator can access.
  • You have the login credentials to access the data source.
  • You are familiar with the creation of custom resource files in YAML format.

Procedure

  1. Open a text editor, create a file with the name of the virtualization, and save it with the extension .yaml, for example, dv-customer.yaml.
  2. Add information to define the custom resource kind, name, and source. The following annotated example provides guidance on the contents to include in the CR:

    dv-customer.yaml

    apiVersion: teiid.io/v1alpha1
    kind: VirtualDatabase
    metadata:
      name: dv-customer
    spec:
      replicas: 1
      env:
      - name: SPRING_DATASOURCE_SAMPLEDB_USERNAME 1
      value: user
      - name: SPRING_DATASOURCE_SAMPLEDB_PASSWORD
      value: mypassword
      - name: SPRING_DATASOURCE_SAMPLEDB_DATABASENAME
      value: sampledb
      - name: SPRING_DATASOURCE_SAMPLEDB_JDBCURL 2
      value: jdbc:postgresql://postgresql/$(SPRING_DATASOURCE_SAMPLEDB_DATABASENAME)
      resources:
        memory: 1024Mi
        cpu: 2.0
      build:
        source:
          maven: com.example:customer-vdb:1.0.0:vdb 3
        mavenRepositories: 4
          central: https://repo.maven.apache.org/maven2

    1
    Specifies the credentials for signing in to the data source. Although this example shows credentials that are defined within the CR, in production use, use secrets to specify credentials, rather than exposing them in plain text. For information about adding credentials to secrets, see xref:
    2
    Specifies the URL for connecting to the data source.
    3
    Specifies the Maven location of the virtual database by providing the groupId, artifactId, and version (GAV) coordinates.
    4
    If you are using a private Maven repository, specify its URL.

After you create the CR YAML file, you can run the Data Virtualization Operator to deploy the virtual database to OpenShift.

Run the Data Virtualization Operator with the CR to generate the virtual database and deploy it to OpenShift.