Chapter 6. Using artifact repositories in a restricted environment

This section describes how to manually configure various technology stacks to work with artifacts from in-house repositories using self-signed certificates.

6.1. Using Maven artifact repositories

Maven downloads artifacts that are defined in two locations:

  • Artifact repositories defined in a pom.xml file of the project. Configuring repositories in pom.xml is not specific to Red Hat CodeReady Workspaces. For more information, see the Maven documentation about the POM.
  • Artifact repositories defined in a settings.xml file. By default, settings.xml is located at `~/.m2/settings.xml.

6.1.1. Defining repositories in settings.xml

To specify your own artifact repositories at example.server.org, use the settings.xml file. To do that, ensure, that settings.xml is present in all the containers that use Maven tools, in particular the Maven container and the Java plug-in container.

By default, settings.xml is located at the <home dir>/.m2 directory which is already on persistent volume in Maven and Java plug-in containers and you don’t need to re-create the file each time you restart the workspace if it isn’t in ephemeral mode.

In case you have another container that uses Maven tools and you want to share <home dir>/.m2 folder with this container, you have to specify the custom volume for this specific component in the devfile: 

apiVersion: 1.0.0
metadata:
  name: MyDevfile
components:
  - type: chePlugin
    alias: maven-tool
    id: plugin/id
    volumes:
    - name: m2
      containerPath: <home dir>/.m2

Procedure

  1. Configure your settings.xml file to use artifact repositories at example.server.org: 

    <settings>
  <profiles>
    <profile>
      <id>my-nexus</id>
      <pluginRepositories>
        <pluginRepository>
           <id>my-nexus-snapshots</id>
           <releases>
             <enabled>false</enabled>
           </releases>
           <snapshots>
             <enabled>true</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-snapshots/</url>
        </pluginRepository>
        <pluginRepository>
           <id>my-nexus-releases</id>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-releases/</url>
        </pluginRepository>
      </pluginRepositories>
      <repositories>
        <repository>
           <id>my-nexus-snapshots</id>
           <releases>
             <enabled>false</enabled>
           </releases>
           <snapshots>
             <enabled>true</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-snapshots/</url>
        </repository>
        <repository>
           <id>my-nexus-releases</id>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-releases/</url>
        </repository>
      </repositories>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>my-nexus</activeProfile>
  </activeProfiles>
</settings>

6.1.2. Defining Maven settings.xml file across workspaces

To use your own settings.xml file across all your workspaces, create a Secret object (with a name of your choice) in the same project as the workspace. Put the contents of the required settings.xml in the data section of the Secret (possibly along with other files that should reside in the same directory). Labelling and annotating this Secret according to Mounting a secret as a file or an environment variable into a workspace container ensures that the contents of the Secret is mounted into the workspace Pod. Note that you need to restart any previously running workspaces for them to use this Secret.

Prerequisites

This is required to set your private credentials to a Maven repository. See the Maven documentation Settings.xml#Servers for additional information.

To mount this settings.xml: 

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                              https://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>repository-id</id>
      <username>username</username>
      <password>password123</password>
    </server>
  </servers>
</settings>

Procedure

  1. Convert settings.xml to base64: 

    $ cat settings.xml | base64

  2. Copy the output to a new file, secret.yaml, which also defines needed annotations and labels: 

    apiVersion: v1
kind: Secret
metadata:
  name: maven-settings-secret
  labels:
    app.kubernetes.io/part-of: che.eclipse.org
    app.kubernetes.io/component: workspace-secret
  annotations:
    che.eclipse.org/target-container: maven
    che.eclipse.org/mount-path: /home/user/.m2
    che.eclipse.org/mount-as: file
type: Opaque
data:
  settings.xml: PHNldHRpbmdzIHhtbG5zPSJodHRwOi8vbWF2ZW4uYXBhY2hlLm9yZy9TRVRUSU5HUy8xLjAuMCIKICAgICAgICAgIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiCiAgICAgICAgICB4c2k6c2NoZW1hTG9jYXRpb249Imh0dHA6Ly9tYXZlbi5hcGFjaGUub3JnL1NFVFRJTkdTLzEuMC4wCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGh0dHBzOi8vbWF2ZW4uYXBhY2hlLm9yZy94c2Qvc2V0dGluZ3MtMS4wLjAueHNkIj4KICA8c2VydmVycz4KICAgIDxzZXJ2ZXI+CiAgICAgIDxpZD5yZXBvc2l0b3J5LWlkPC9pZD4KICAgICAgPHVzZXJuYW1lPnVzZXJuYW1lPC91c2VybmFtZT4KICAgICAgPHBhc3N3b3JkPnBhc3N3b3JkMTIzPC9wYXNzd29yZD4KICAgIDwvc2VydmVyPgogIDwvc2VydmVycz4KPC9zZXR0aW5ncz4K

As of {7-15-2-2} version of CodeReady Workspaces, the target container annotation has been deprecated. An example of the updated annotation looks as follows:

+ 

apiVersion: v1
kind: Secret
metadata:
  name: maven-settings-secret
  labels:
    app.kubernetes.io/part-of: che.eclipse.org
    app.kubernetes.io/component: workspace-secret
  annotations:
    che.eclipse.org/automount-workspace-secret: true
    che.eclipse.org/mount-path: /home/user/.m2
    che.eclipse.org/mount-as: file
type: Opaque
data:
  settings.xml: PHNldHRpbmdzIHhtbG5zPSJodHRwOi8vbWF2ZW4uYXBhY2hlLm9yZy9TRVRUSU5HUy8xLjAuMCIKICAgICAgICAgIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiCiAgICAgICAgICB4c2k6c2NoZW1hTG9jYXRpb249Imh0dHA6Ly9tYXZlbi5hcGFjaGUub3JnL1NFVFRJTkdTLzEuMC4wCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGh0dHBzOi8vbWF2ZW4uYXBhY2hlLm9yZy94c2Qvc2V0dGluZ3MtMS4wLjAueHNkIj4KICA8c2VydmVycz4KICAgIDxzZXJ2ZXI+CiAgICAgIDxpZD5yZXBvc2l0b3J5LWlkPC9pZD4KICAgICAgPHVzZXJuYW1lPnVzZXJuYW1lPC91c2VybmFtZT4KICAgICAgPHBhc3N3b3JkPnBhc3N3b3JkMTIzPC9wYXNzd29yZD4KICAgIDwvc2VydmVyPgogIDwvc2VydmVycz4KPC9zZXR0aW5ncz4K

  1. Create this secret in the cluster: 

    $ oc apply -f secret.yaml
  2. Start a new workspace. You will see /home/user/.m2/settings.xml with your original content in the maven container.

6.1.3. Using self-signed certificates in Java projects

Internal artifact repositories often do not have a certificate signed by an authority that is trusted by default in Java. They are usually signed by an internal company authority or are self-signed. Configure your tools to accept these certificates by adding them to the Java truststore.

Procedure

  1. Obtain a server certificate file from the repository server. It is often a file named tls.crt.

    1. Create a Java truststore file: 

      $ keytool -import -file tls.crt -alias nexus -keystore truststore.jks -storepass changeit

Trust this certificate? [no]:  yes
Certificate was added to keystore
Owner: CN=example.com
Issuer: CN=example.com
Serial number: 80ca0f6980c6019a
Valid from: Thu Feb 06 11:00:29 CET 2020 until: Fri Feb 05 11:00:29 CET 2021
Certificate fingerprints:
     MD5:  88:3C:EC:E1:BE:57:DD:9D:46:36:8E:DD:BF:14:04:22
     SHA1: 08:D8:79:D3:F8:6B:5C:3D:71:AA:23:CA:72:01:47:BD:9D:91:0A:AD
     SHA256: 5C:BB:66:81:44:D2:50:EE:EB:CE:D6:15:7E:63:E1:9A:71:EA:58:3F:14:01:15:4E:68:5D:71:0A:A0:31:33:29
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 4096-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: *.apps.example.com
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
    2. Upload the truststore file to /projects/maven/truststore.jks to make it available for all containers.

  2. Add the truststore file.

    • In the Maven container:

      1. Add the javax.net.ssl system property to the MAVEN_OPTS environment variable: 

          - mountSources: true
    alias: maven
    type: dockerimage
    ...
    env:
       -name: MAVEN_OPTS
        value: >-
          -Duser.home=/projects/maven -Djavax.net.ssl.trustStore=/projects/truststore.jks
      2. Restart the workspace.

    • In the Java plug-in container:

      In the devfile, add the javax.net.ssl system property for the Java language server: 

      components:
  - id: redhat/java11/latest
    type: chePlugin
    preferences:
      java.jdt.ls.vmargs: >-
        -noverify -Xmx1G -XX:+UseG1GC -XX:+UseStringDeduplication
        -Duser.home=/projects/maven
        -Djavax.net.ssl.trustStore=/projects/truststore.jks
[...]

6.2. Using Gradle artifact repositories

6.2.1. Downloading different versions of Gradle

The recommended way to download any version of Gradle is by using the Gradle Wrapper script. If your project does not have a gradle/wrapper directory, run $ gradle wrapper to configure the Wrapper.

Prerequisites

  • The Gradle Wrapper is present in your project.

Procedure

To download a Gradle version from a non-standard location, change your Wrapper settings in /projects/<your_project>/gradle/wrapper/gradle-wrapper.properties:

  • Change the distributionUrl property to point to a URL of the Gradle distribution ZIP file: 

    properties
distributionUrl=http://<url_to_gradle>/gradle-6.1-bin.zip

Alternatively, you may place a Gradle distribution zip file locally in /project/gradle in your workspace.

  • Change the distributionUrl property to point to a local address of the Gradle distribution zip file: 

    properties
distributionUrl=file\:/projects/gradle/gradle-6.1-bin.zip

6.2.2. Configuring global Gradle repositories

Use an initialization script to configure global repositories for the workspace. Gradle performs extra configuration before projects are evaluated, and this configuration is used in each Gradle project from the workspace.

Procedure

To set global repositories for Gradle that could be used in each Gradle project in the workspace, create an init.gradle script in the ~/.gradle/ directory: 

allprojects {
  repositories {
    mavenLocal ()
    maven {
      url "http://repo.mycompany.com/maven"
      credentials {
        username "admin"
        password "my_password"
      }
    }
  }
}

This file configures Gradle to use a local Maven repository with the given credentials.

Note

The ~/.gradle directory does not persist in the current Java plug-in versions, so you must create the init.gradle script at each workspace start in the Java plug-in sidecar container.

6.2.3. Using self-signed certificates in Java projects

Internal artifact repositories often do not have a certificate signed by an authority that is trusted by default in Java. They are usually signed by an internal company authority or are self-signed. Configure your tools to accept these certificates by adding them to the Java truststore.

Procedure

  1. Obtain a server certificate file from the repository server. It is often a file named tls.crt.

    1. Create a Java truststore file: 

      $ keytool -import -file tls.crt -alias nexus -keystore truststore.jks -storepass changeit

Trust this certificate? [no]:  yes
Certificate was added to keystore
Owner: CN=example.com
Issuer: CN=example.com
Serial number: 80ca0f6980c6019a
Valid from: Thu Feb 06 11:00:29 CET 2020 until: Fri Feb 05 11:00:29 CET 2021
Certificate fingerprints:
     MD5:  88:3C:EC:E1:BE:57:DD:9D:46:36:8E:DD:BF:14:04:22
     SHA1: 08:D8:79:D3:F8:6B:5C:3D:71:AA:23:CA:72:01:47:BD:9D:91:0A:AD
     SHA256: 5C:BB:66:81:44:D2:50:EE:EB:CE:D6:15:7E:63:E1:9A:71:EA:58:3F:14:01:15:4E:68:5D:71:0A:A0:31:33:29
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 4096-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: *.apps.example.com
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
    2. Upload the truststore file to /projects/gradle/truststore.jks to make it available for all containers.

  2. Add the truststore file in the Gradle container.

    1. Add the javax.net.ssl system property to the JAVA_OPTS environment variable: 

        - mountSources: true
    alias: maven
    type: dockerimage
    ...
    env:
       -name: JAVA_OPTS
        value: >-
          -Duser.home=/projects/gradle -Djavax.net.ssl.trustStore=/projects/truststore.jks

Additional resources

6.3. Using Python artifact repositories

6.3.1. Configuring Python to use a non-standard registry

To specify a non-standard repository for use by the Python pip tool, set the PIP_INDEX_URL environment variable.

Procedure

  • In your devfile, configure the PIP_INDEX_URL environment variable for the language support and for the development container components: 

      - id: ms-python/python/latest
    memoryLimit: 512Mi
    type: chePlugin
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
  - mountSources: true
    memoryLimit: 512Mi
    type: dockerimage
    alias: python
    image: 'quay.io/eclipse/che-python-3.7:nightly'
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'

6.3.2. Using self-signed certificates in Python projects

Internal artifact repositories often do not have a self-signed (SSL) certificate signed by an authority that is trusted by default. They are usually signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Python uses certificates from a file defined in the PIP_CERT environment variable.

Procedure

  1. Obtain the certificate from the non-standard repository and place the certificate file in the /projects/ssl file to make it accessible from all your containers.

    Note

    pip accepts certificates in the Privacy-Enhanced Mail (PEM) format only. Convert the certificate to the PEM format using OpenSSL if necessary.

  2. Configure the devfile: 

     - id: ms-python/python/latest
    memoryLimit: 512Mi
    type: chePlugin
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
      - value: '/projects/ssl/rootCA.pem'
        name: 'PIP_CERT'
  - mountSources: true
    memoryLimit: 512Mi
    type: dockerimage
    alias: python
    image: 'quay.io/eclipse/che-python-3.7:nightly'
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
      - value:'/projects/ssl/rootCA.pem'
        name: 'PIP_CERT'

6.4. Using Go artifact repositories

To configure Go in a restricted environment, use the GOPROXY environment variable and the Athens module datastore and proxy.

6.4.1. Configuring Go to use a non-standard-registry

Athens is a Go module datastore and proxy with many configuration options. It can be configured to act only as a module datastore and not as a proxy. An administrator can upload their Go modules to the Athens datastore and have them available across their Go projects. If a project tries to access a Go module that is not in the Athens datastore, the Go build fails.

  • To work with Athens, configure the GOPROXY environment variable in the devfile of your CLI container: 

    components:
- mountSources: true
  type: dockerimage
  alias: go-cli
  image: 'quay.io/eclipse/che-golang-1.12:7.7.0'
  ...
  - value: /tmp/.cache
    name: GOCACHE
  - value: 'http://your.athens.host'
    name: GOPROXY

6.4.2. Using self-signed certificates in Go projects

Internal artifact repositories often do not have a self-signed (SSL) certificate signed by an authority that is trusted by default. They are usually signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Go uses certificates from a file defined in the SSL_CERT_FILE environment variable.

Procedure

  1. Obtain the certificate used by the Athens server in the Privacy-Enhanced Mail (PEM) format and place it in the /projects/ssl file to make it accessible from all your containers.
  2. Right-click the project explorer and select Upload files to upload the rootCA.crt certificate file to your Red Hat CodeReady Workspaces workspace.

  3. Add the appropriate environment variables to your devfile: 

    components:
- mountSources: true
  type: dockerimage
  alias: go-cli
  image: 'quay.io/eclipse/che-golang-1.12:7.7.0'
  ...
  - value: /tmp/.cache
    name: GOCACHE
  - value: 'http://your.athens.host'
    name: GOPROXY
  - value: 'on'
    name: GO111MODULE
  - value: '/projects/ssl/rootCA.crt'
    name: SSL_CERT_FILE

Additional resources

6.5. Using NuGet artifact repositories

To configure NuGet in a restricted environment, modify the nuget.config file and use the SSL_CERT_FILE environment variable in the devfile to add self-signed certificates.

6.5.1. Configuring NuGet to use a non-standard artifact repository

NuGet searches for configuration files anywhere between the solution directory and the driver root directory. If you put the nuget.config file in the /projects directory, the nuget.config file defines NuGet behavior for all projects in /projects.

Procedure

  • Create and place the nuget.config file in the /projects directory.

    Example nuget.config with a Nexus repository hosted at nexus.example.org:

    <?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <packageSources>
    <add key="nexus2" value="https://nexus.example.org/repository/nuget-hosted/"/>
  </packageSources>
  <packageSourceCredentials>
    <nexus2>
        <add key="Username" value="user" />
        <add key="Password" value="..." />
    </nexus2>
  </packageSourceCredentials>
</configuration>

6.5.2. Using self-signed certificates in NuGet projects

Internal artifact repositories often do not have a self-signed (SSL) certificate signed by an authority that is trusted by default. They are usually signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Procedure

  1. Obtain the certificate file of a non-standard repository and place it in the /projects/ssl file to make it accessible from all your containers.

  2. Specify the location of the certificate file in the SSL_CERT_FILE environment variable in your devfile for the OmniSharp plug-in and for the .NET container.

    Example of the devfile:

    components:
  - id: redhat-developer/che-omnisharp-plugin/latest
    memoryLimit: 1024Mi
    type: chePlugin
    alias: omnisharp
    env:
     - value: /projects/ssl/rootCA.crt
       name: SSL_CERT_FILE
 - mountSources: true
   endpoints:
     - name: 5000/tcp
       port: 5000
   memoryLimit: 512Mi
   type: dockerimage
   volumes:
     - name: dotnet
       containerPath: /home/user
   alias: dotnet
   image: 'quay.io/eclipse/che-dotnet-2.2:7.7.1'
   env:
     - value: /projects/ssl/rootCA.crt
       name: SSL_CERT_FILE

6.6. Using npm artifact repositories

npm is usually configured using the npm config command, writing values to the .npmrc files. However, configuration values can also be set using the environment variables beginning with NPM_CONFIG_.

The Javascript/Typescript plug-in used in Red Hat CodeReady Workspaces does not download any artifacts. It is enough to configure npm in the dev-machine component.

Use the following environment variables for configuration:

  • The URL for the artifact repository: NPM_CONFIG_REGISTRY
  • For using a certificate from a file: NODE_EXTRA_CA_CERTS

To be able to reference the certificate in a devfile, get a copy of the certificate of the npm repository server and put it inside the /project folder.

  1. An example configuration for the use of an internal repository with a self-signed certificate: 

      - mountSources: true
    endpoints:
      - name: nodejs
        port: 3000
    memoryLimit: '512Mi'
    type: 'dockerimage'
    alias: 'nodejs'
    image: 'quay.io/eclipse/che-nodejs10-ubi:nightly'
    env:
      -name: NODE_EXTRA_CA_CERTS
       value: '/projects/config/tls.crt'
     - name: NPM_CONFIG_REGISTRY
       value: 'https://snexus-airgap.apps.acme.com/repository/npm-proxy/'