Chapter 6. Using artifact repositories in a restricted environment

By configuring technology stacks, you can work with artifacts from in-house repositories using self-signed certificates.

6.1. Using Maven artifact repositories

Maven downloads artifacts that are defined in two locations:

  • Artifact repositories defined in a pom.xml file of the project. Configuring repositories in pom.xml is not specific to Red Hat CodeReady Workspaces. For more information, see the Maven documentation about the POM.
  • Artifact repositories defined in a settings.xml file. By default, settings.xml is located at `~/.m2/settings.xml.

6.1.1. Defining repositories in settings.xml

To specify your own artifact repositories at example.server.org, use the settings.xml file. Ensure that settings.xml is in all the containers that use Maven tools. In particular, ensure that it is in the Maven container and the Java plug-in container.

By default, settings.xml is located at the <home dir>/.m2 directory which is already on persistent volume in Maven and Java plug-in containers and you don’t need to re-create the file each time you restart the workspace if it isn’t in ephemeral mode.

In case you have another container that uses Maven tools and you are about to share <home dir>/.m2 folder with this container, you have to specify the custom volume for this specific component in the devfile: 

apiVersion: 1.0.0
metadata:
  name: MyDevfile
components:
  - type: chePlugin
    alias: maven-tool
    id: plugin/id
    volumes:
    - name: m2
      containerPath: <home dir>/.m2

Procedure

  1. Configure your settings.xml file to use artifact repositories at example.server.org: 

    <settings>
  <profiles>
    <profile>
      <id>my-nexus</id>
      <pluginRepositories>
        <pluginRepository>
           <id>my-nexus-snapshots</id>
           <releases>
             <enabled>false</enabled>
           </releases>
           <snapshots>
             <enabled>true</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-snapshots/</url>
        </pluginRepository>
        <pluginRepository>
           <id>my-nexus-releases</id>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-releases/</url>
        </pluginRepository>
      </pluginRepositories>
      <repositories>
        <repository>
           <id>my-nexus-snapshots</id>
           <releases>
             <enabled>false</enabled>
           </releases>
           <snapshots>
             <enabled>true</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-snapshots/</url>
        </repository>
        <repository>
           <id>my-nexus-releases</id>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
           <url>http://example.server.org/repository/maven-releases/</url>
        </repository>
      </repositories>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>my-nexus</activeProfile>
  </activeProfiles>
</settings>

6.1.2. Defining Maven settings.xml file across workspaces

To use your own settings.xml file across all your workspaces, create a Secret object (with a name of your choice) in the same project as the workspace. Put the contents of the required settings.xml in the data section of the Secret (possibly along with other files that should reside in the same directory). Labelling and annotating this Secret according to Section 3.10.1, “Mounting a secret as a file into a workspace container” ensures that the contents of the Secret is mounted into the workspace Pod. Note that you need to restart any previously running workspaces for them to use this Secret.

Prerequisites

This is required to set your private credentials to a Maven repository. See the Maven documentation Settings.xml#Servers for additional information.

To mount this settings.xml: 

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                              https://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>repository-id</id>
      <username>username</username>
      <password>password123</password>
    </server>
  </servers>
</settings>

Procedure

  1. Convert settings.xml to base64: 

    $ cat settings.xml | base64

  2. Copy the output to a new file, secret.yaml, which also defines needed annotations and labels: 

    apiVersion: v1
kind: Secret
metadata:
  name: maven-settings-secret
  labels:
    app.kubernetes.io/part-of: che.eclipse.org
    app.kubernetes.io/component: workspace-secret
  annotations:
    che.eclipse.org/automount-workspace-secret: "true"
    che.eclipse.org/mount-path: /home/jboss/.m2
    che.eclipse.org/mount-as: file
type: Opaque
data:
  settings.xml: PHNldHRpbmdzIHhtbG5zPSJodHRwOi8vbWF2ZW4uYXBhY2hlLm9yZy9TRVRUSU5HUy8xLjAuMCIKICAgICAgICAgIHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiCiAgICAgICAgICB4c2k6c2NoZW1hTG9jYXRpb249Imh0dHA6Ly9tYXZlbi5hcGFjaGUub3JnL1NFVFRJTkdTLzEuMC4wCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGh0dHBzOi8vbWF2ZW4uYXBhY2hlLm9yZy94c2Qvc2V0dGluZ3MtMS4wLjAueHNkIj4KICA8c2VydmVycz4KICAgIDxzZXJ2ZXI+CiAgICAgIDxpZD5yZXBvc2l0b3J5LWlkPC9pZD4KICAgICAgPHVzZXJuYW1lPnVzZXJuYW1lPC91c2VybmFtZT4KICAgICAgPHBhc3N3b3JkPnBhc3N3b3JkMTIzPC9wYXNzd29yZD4KICAgIDwvc2VydmVyPgogIDwvc2VydmVycz4KPC9zZXR0aW5ncz4K

  3. Create this secret in the cluster: 

    $ oc  apply -f secret.yaml
  4. Start a new workspace. The maven container contains a file /home/jboss/.m2/settings.xml with your original content.

6.1.2.1. OpenShift 3.11 and OpenShift <1.13

On OpenShift 3.11 , it’s impossible to have multiple VolumeMounts at same path so having devfile with volume /home/jboss/.m2 and secret at /home/jboss/.m2/settings.xml would resolve into the conflict. On these clusters use /home/jboss/.m2/repository as a volume for maven repository in the devfile: 

apiVersion: 1.0.0
metadata:
  ...
components:
 - type: dockerimage
   alias: maven
   image: maven:3.11
   volumes:
     - name: m2
       containerPath: /home/jboss/.m2/repository
   ...

6.1.3. Using self-signed certificates in Maven projects

Internal artifact repositories often do not have a certificate signed by an authority that is trusted by default in Java. They are mainly signed by an internal company authority or are self-signed. Configure your tools to accept these certificates by adding them to the Java truststore.

Procedure

  1. Obtain a server certificate file from the repository server. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

    1. Copy the original Java truststore file: 

      $ mkdir /projects/maven
$ cp $JAVA_HOME/lib/security/cacerts /projects/maven/truststore.jks
$ chmod +w /projects/maven/truststore.jks

    2. Import the certificate into the Java truststore file 

      $ keytool -import -noprompt -file /public-certs/nexus.cer -alias nexus -keystore /projects/maven/truststore.jks -storepass changeit
Certificate was added to keystore

  2. Add the truststore file.

    • In the Maven container:

      1. Add the javax.net.ssl system property to the MAVEN_OPTS environment variable: 

          - mountSources: true
    alias: maven
    type: dockerimage
    ...
    env:
       -name: MAVEN_OPTS
        value: >-
          -Duser.home=/projects/maven -Djavax.net.ssl.trustStore=/projects/maven/truststore.jks -Djavax.net.ssl.trustStorePassword=changeit
      2. Restart the workspace.

    • In the Java plug-in container:

      In the devfile, add the javax.net.ssl system property for the Java language server: 

      components:
  - id: redhat/java11/latest
    type: chePlugin
    preferences:
      java.jdt.ls.vmargs: >-
        -noverify -Xmx1G -XX:+UseG1GC -XX:+UseStringDeduplication
        -Duser.home=/projects/maven
        -Djavax.net.ssl.trustStore=/projects/maven/truststore.jks
        -Djavax.net.ssl.trustStorePassword=changeit
...

6.2. Using Gradle artifact repositories

This section describes how to download and configure Gradle.

6.2.1. Downloading different versions of Gradle

The recommended way to download any version of Gradle is by using the Gradle Wrapper script. If your project does not have a gradle/wrapper directory, run $ gradle wrapper to configure the Wrapper.

Prerequisites

  • The Gradle Wrapper is available in your project.

Procedure

To download a Gradle version from a non-standard location, change your Wrapper settings in /projects/<your_project>/gradle/wrapper/gradle-wrapper.properties:

  • Change the distributionUrl property to point to a URL of the Gradle distribution ZIP file: 

    properties
distributionUrl=http://<url_to_gradle>/gradle-6.1-bin.zip

Alternatively, you may place a Gradle distribution zip file locally in /project/gradle in your workspace.

  • Change the distributionUrl property to point to a local address of the Gradle distribution zip file: 

    properties
distributionUrl=file\:/projects/gradle/gradle-6.1-bin.zip

6.2.2. Configuring global Gradle repositories

Use an initialization script to configure global repositories for the workspace. Gradle performs extra configuration before projects are evaluated, and this configuration is used in each Gradle project from the workspace.

Procedure

To set global repositories for Gradle that could be used in each Gradle project in the workspace, create an init.gradle script in the ~/.gradle/ directory: 

allprojects {
  repositories {
    mavenLocal ()
    maven {
      url "http://repo.mycompany.com/maven"
      credentials {
        username "admin"
        password "my_password"
      }
    }
  }
}

This file configures Gradle to use a local Maven repository with the given credentials.

Note

The ~/.gradle directory does not persist in the current Java plug-in versions, so you must create the init.gradle script at each workspace start in the Java plug-in sidecar container.

6.2.3. Using self-signed certificates in Gradle projects

Internal artifact repositories often do not have a certificate signed by an authority that is trusted by default in Java. They are mainly signed by an internal company authority or are self-signed. Configure your tools to accept these certificates by adding them to the Java truststore.

Procedure

  1. Obtain a server certificate file from the repository server. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

    1. Copy the original Java truststore file: 

      $ mkdir /projects/maven
$ cp $JAVA_HOME/lib/security/cacerts /projects/maven/truststore.jks
$ chmod +w /projects/maven/truststore.jks

    2. Import the certificate into the Java truststore file 

      $ keytool -import -noprompt -file /public-certs/nexus.cer -alias nexus -keystore /projects/maven/truststore.jks -storepass changeit
Certificate was added to keystore
    3. Upload the truststore file to /projects/gradle/truststore.jks to make it available for all containers.

  2. Add the truststore file in the Gradle container.

    1. Add the javax.net.ssl system property to the JAVA_OPTS environment variable: 

        - mountSources: true
    alias: maven
    type: dockerimage
    ...
    env:
       -name: JAVA_OPTS
        value: >-
          -Duser.home=/projects/gradle
          -Djavax.net.ssl.trustStore=/projects/maven/truststore.jks
          -Djavax.net.ssl.trustStorePassword=changeit

Additional resources

6.3. Using Python artifact repositories

6.3.1. Configuring Python to use a non-standard registry

To specify a non-standard repository for use by the Python pip tool, set the PIP_INDEX_URL environment variable.

Procedure

  • In your devfile, configure the PIP_INDEX_URL environment variable for the language support and for the development container components: 

      - id: ms-python/python/latest
    memoryLimit: 512Mi
    type: chePlugin
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
  - mountSources: true
    memoryLimit: 512Mi
    type: dockerimage
    alias: python
    image: 'registry.redhat.io/codeready-workspaces/plugin-java8-rhel8:2.5'
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'

6.3.2. Using self-signed certificates in Python projects

Internal artifact repositories often do not have a self-signed TLS certificate signed by an authority that is trusted by default. They are mainly signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Python uses certificates from a file defined in the PIP_CERT environment variable.

Procedure

  1. Obtain the certificate used by the pip server in the Privacy-Enhanced Mail (PEM) format. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

    Note

    pip accepts certificates in the Privacy-Enhanced Mail (PEM) format only. Convert the certificate to the PEM format using OpenSSL if necessary.

  2. Configure the devfile: 

     - id: ms-python/python/latest
    memoryLimit: 512Mi
    type: chePlugin
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
      - value: '/projects/tls/rootCA.pem'
        name: 'PIP_CERT'
  - mountSources: true
    memoryLimit: 512Mi
    type: dockerimage
    alias: python
    image: 'registry.redhat.io/codeready-workspaces/plugin-java8-rhel8:2.5'
    env:
      - name: 'PIP_INDEX_URL'
        value: 'https://<username>:<password>@pypi.company.com/simple'
      - value: '/projects/tls/rootCA.pem'
        name: 'PIP_CERT'

6.4. Using Go artifact repositories

To configure Go in a restricted environment, use the GOPROXY environment variable and the Athens module data store and proxy.

6.4.1. Configuring Go to use a non-standard-registry

Athens is a Go module data store and proxy with many configuration options. It can be configured to act only as a module data store and not as a proxy. An administrator can upload their Go modules to the Athens data store and have them available across their Go projects. If a project tries to access a Go module that is not in the Athens data store, the Go build fails.

  • To work with Athens, configure the GOPROXY environment variable in the devfile of your CLI container: 

    components:
- mountSources: true
  type: dockerimage
  alias: go-cli
  image: 'quay.io/eclipse/che-golang-1.12:7.7.0'
  ...
  - value: /tmp/.cache
    name: GOCACHE
  - value: 'http://your.athens.host'
    name: GOPROXY

6.4.2. Using self-signed certificates in Go projects

Internal artifact repositories often do not have a self-signed TLS certificate signed by an authority that is trusted by default. They are typically signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Go uses certificates from a file defined in the SSL_CERT_FILE environment variable.

Procedure

  1. Obtain the certificate used by the Athens server in the Privacy-Enhanced Mail (PEM) format. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

  2. Add the appropriate environment variables to your devfile: 

    components:
- mountSources: true
  type: dockerimage
  alias: go-cli
  image: 'registry.redhat.io/codeready-workspaces/stacks-golang-rhel8:2.5'
  ...
  - value: /tmp/.cache
    name: GOCACHE
  - value: 'http://your.athens.host'
    name: GOPROXY
  - value: '/projects/tls/rootCA.crt'
    name: SSL_CERT_FILE

Additional resources

6.5. Using NuGet artifact repositories

To configure NuGet in a restricted environment, modify the nuget.config file and use the SSL_CERT_FILE environment variable in the devfile to add self-signed certificates.

6.5.1. Configuring NuGet to use a non-standard artifact repository

NuGet searches for configuration files anywhere between the solution directory and the driver root directory. If you put the nuget.config file in the /projects directory, the nuget.config file defines NuGet behavior for all projects in /projects.

Procedure

  • Create and place the nuget.config file in the /projects directory.

    Example nuget.config with a Nexus repository hosted at nexus.example.org:

    <?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <packageSources>
    <add key="nexus2" value="https://nexus.example.org/repository/nuget-hosted/"/>
  </packageSources>
  <packageSourceCredentials>
    <nexus2>
        <add key="Username" value="user" />
        <add key="Password" value="..." />
    </nexus2>
  </packageSourceCredentials>
</configuration>

6.5.2. Using self-signed certificates in NuGet projects

Internal artifact repositories often do not have a self-signed TLS certificate signed by an authority that is trusted by default. They are mainly signed by an internal company authority or are self-signed. Configure your tools to accept these certificates.

Procedure

  1. Obtain the certificate used by the .NET server in the Privacy-Enhanced Mail (PEM) format. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

  2. Specify the location of the certificate file in the SSL_CERT_FILE environment variable in your devfile for the OmniSharp plug-in and for the .NET container.

    Example of the devfile:

    components:
  - id: redhat-developer/che-omnisharp-plugin/latest
    memoryLimit: 1024Mi
    type: chePlugin
    alias: omnisharp
    env:
     - value: /public-certs/nuget.cer
       name: SSL_CERT_FILE
 - mountSources: true
   endpoints:
     - name: 5000/tcp
       port: 5000
   memoryLimit: 512Mi
   type: dockerimage
   volumes:
     - name: dotnet
       containerPath: /home/jboss
   alias: dotnet
   image: 'quay.io/eclipse/che-dotnet-2.2:7.7.1'
   env:
     - value: /projects/tls/rootCA.crt
       name: SSL_CERT_FILE

6.6. Using npm artifact repositories

The npm (Node Package Manager) package manager for the JavaScript programming language is configured using the npm config command, by writing values to the .npmrc files. However, configuration values can also be set using the environment variables beginning with NPM_CONFIG_.

The Typescript plug-in used in Red Hat CodeReady Workspaces does not download any artifacts. It is enough to configure npm in the dev-machine component.

Use the following environment variables for configuration:

  • The URL for the artifact repository: NPM_CONFIG_REGISTRY
  • For using a certificate from a file: NODE_EXTRA_CA_CERTS

Obtain a server certificate file from the repository server. It is customary for administrators to provide certificates of internal artifact servers as OpenShift secrets (see Importing untrusted TLS certificates to CodeReady Workspaces). The relevant server certificates will be mounted in /public-certs in every container in the workspace.

  1. An example configuration for the use of an internal repository with a self-signed certificate: 

      - mountSources: true
    endpoints:
      - name: nodejs
        port: 3000
    memoryLimit: '512Mi'
    type: 'dockerimage'
    alias: 'nodejs'
    image: 'quay.io/eclipse/che-nodejs10-ubi:nightly'
    env:
      -name: NODE_EXTRA_CA_CERTS
       value: '/public-certs/nexus.cer
     - name: NPM_CONFIG_REGISTRY
       value: 'https://snexus-airgap.apps.acme.com/repository/npm-proxy/'