Developer Guide

OpenShift Online 3

OpenShift Online Developer Reference

Red Hat OpenShift Documentation Team

Abstract

These topics help developers set up and configure a workstation to develop and deploy applications in an OpenShift Online cloud environment with a command-line interface (CLI). This guide provide s detailed instructions and examples to help developers:

Chapter 1. Overview

This guide is intended for application developers, and provides instructions for setting up and configuring a workstation to develop and deploy applications in an OpenShift Online cloud environment. This includes detailed instructions and examples to help developers:

Chapter 2. Application Life Cycle Management

2.1. Planning Your Development Process

2.1.1. Overview

OpenShift Online is designed for building and deploying applications. Depending on how much you want to involve OpenShift Online in your development process, you can choose to:

  • focus your development within an OpenShift Online project, using it to build an application from scratch then continuously develop and manage its lifecycle, or
  • bring an application (e.g., binary, container image, source code) you have already developed in a separate environment and deploy it onto OpenShift Online.

2.1.2. Using OpenShift Online as Your Development Environment

Deploying then developing on OpenShift Online

You can begin your application’s development from scratch using OpenShift Online directly. Consider the following steps when planning this type of development process:

Initial Planning

  • What does your application do?
  • What programming language will it be developed in?

Access to OpenShift Online

Develop

  • Using your editor or IDE of choice, create a basic skeleton of an application. It should be developed enough to tell OpenShift Online what kind of application it is.
  • Push the code to your Git repository.

Generate

Manage

  • Start developing your application code.
  • Ensure your application builds successfully.
  • Continue to locally develop and polish your code.
  • Push your code to a Git repository.
  • Is any extra configuration needed? Explore the Developer Guide for more options.

Verify

  • You can verify your application in a number of ways. You can push your changes to your application’s Git repository, and use OpenShift Online to rebuild and redeploy your application. Alternatively, you can hot deploy using rsync to synchronize your code changes into a running pod.

2.1.3. Bringing an Application to Deploy on OpenShift Online

Developing then deploying on OpenShift Online

Another possible application development strategy is to develop locally, then use OpenShift Online to deploy your fully developed application. Use the following process if you plan to have application code already, then want to build and deploy onto an OpenShift Online installation when completed:

Initial Planning

  • What does your application do?
  • What programming language will it be developed in?

Develop

  • Develop your application code using your editor or IDE of choice.
  • Build and test your application code locally.
  • Push your code to a Git repository.

Access to OpenShift Online

Generate

Verify

  • Ensure that the application that you have built and deployed in the above Generate step is successfully running on OpenShift Online.

Manage

  • Continue to develop your application code until you are happy with the results.
  • Rebuild your application in OpenShift Online to accept any newly pushed code.
  • Is any extra configuration needed? Explore the Developer Guide for more options.

2.2. Creating New Applications

2.2.1. Overview

You can create a new OpenShift Online application from components including source or binary code, images and/or templates by using either the OpenShift CLI or web console.

2.2.2. Creating an Application Using the CLI

2.2.2.1. Creating an Application From Source Code

The new-app command allows you to create applications from source code in a local or remote Git repository.

To create an application using a Git repository in a local directory:

$ oc new-app /path/to/source/code
Note

If using a local Git repository, the repository should have a remote named origin that points to a URL accessible by the OpenShift Online cluster. If there is no recognised remote, new-app will create a binary build.

You can use a subdirectory of your source code repository by specifying a --context-dir flag. To create an application using a remote Git repository and a context subdirectory:

$ oc new-app https://github.com/openshift/sti-ruby.git \
    --context-dir=2.0/test/puma-test-app

Also, when specifying a remote URL, you can specify a Git branch to use by appending #<branch_name> to the end of the URL:

$ oc new-app https://github.com/openshift/ruby-hello-world.git#beta4

The new-app command creates a build configuration, which itself creates a new application image from your source code. The new-app command typically also creates a deployment configuration to deploy the new image, and a service to provide load-balanced access to the deployment running your image.

OpenShift Online automatically detects whether the Pipeline or Sourcebuild strategy should be used, and in the case of Source builds, detects an appropriate language builder image.

Build Strategy Detection

If a Jenkinsfile exists in the root or specified context directory of the source repository when creating a new application, OpenShift Online generates a Pipeline build strategy. Otherwise, it generates a Source strategy.

You can override the build strategy by setting the --strategy flag to either pipeline or source.

$ oc new-app /home/user/code/myapp --strategy=source

Language Detection

If using the Source build strategy, new-app attempts to determine the language builder to use by the presence of certain files in the root or specified context directory of the repository:

Table 2.1. Languages Detected by new-app

LanguageFiles

dotnet

project.json, *.csproj

jee

pom.xml

nodejs

app.json, package.json

perl

cpanfile, index.pl

php

composer.json, index.php

python

requirements.txt, setup.py

ruby

Gemfile, Rakefile, config.ru

scala

build.sbt

golang

Godeps, main.go

After a language is detected, new-app searches the OpenShift Online server for image stream tags that have a supports annotation matching the detected language, or an image stream that matches the name of the detected language. If a match is not found, new-app searches the Docker Hub registry for an image that matches the detected language based on name.

You can override the image the builder uses for a particular source repository by specifying the image (either an image stream or container specification) and the repository, with a ~ as a separator. Note that if this is done, build strategy detection and language detection are not carried out.

For example, to use the myproject/my-ruby image stream with the source in a remote repository:

$ oc new-app myproject/my-ruby~https://github.com/openshift/ruby-hello-world.git

To use the openshift/ruby-20-centos7:latest container image stream with the source in a local repository:

$ oc new-app openshift/ruby-20-centos7:latest~/home/user/code/my-ruby-app

2.2.2.2. Creating an Application From an Image

You can deploy an application from an existing image. Images can come from image streams in the OpenShift Online server, images in a specific registry or Docker Hub registry, or images in the local Docker server.

The new-app command attempts to determine the type of image specified in the arguments passed to it. However, you can explicitly tell new-app whether the image is a Docker image (using the --docker-image argument) or an image stream (using the -i|--image argument).

Note

If you specify an image from your local Docker repository, you must ensure that the same image is available to the OpenShift Online cluster nodes.

For example, to create an application from the DockerHub MySQL image:

$ oc new-app mysql

To create an application using an image in a private registry, specify the full Docker image specification:

$ oc new-app myregistry:5000/example/myimage
Note

If the registry containing the image is not cluster administrators must ensure that the Docker daemon on the OpenShift Online node hosts is run with the --insecure-registry flag pointing to that registry. You must also tell new-app that the image comes from an insecure registry with the --insecure-registry flag.

You can create an application from an existing image stream and optional image stream tag:

$ oc new-app my-stream:v1

2.2.2.3. Creating an Application From a Template

You can create an application from a previously stored template or from a template file, by specifying the name of the template as an argument. For example, you can store a sample application template and use it to create an application.

To create an application from a stored template:

$ oc create -f examples/sample-app/application-template-stibuild.json
$ oc new-app ruby-helloworld-sample

To directly use a template in your local file system, without first storing it in OpenShift Online, use the -f|--file argument:

$ oc new-app -f examples/sample-app/application-template-stibuild.json

Template Parameters

When creating an application based on a template, use the -p|--param argument to set parameter values defined by the template:

$ oc new-app ruby-helloworld-sample \
    -p ADMIN_USERNAME=admin -p ADMIN_PASSWORD=mypassword

You can store your parameters in a file, then use that file with --param-file when instantiating a template. If you want to read the parameters from standard input, use --param-file=-:

$ cat helloworld.params
ADMIN_USERNAME=admin
ADMIN_PASSWORD=mypassword
$ oc new-app ruby-helloworld-sample --param-file=helloworld.params
$ cat helloworld.params | oc new-app ruby-helloworld-sample --param-file=-

2.2.2.4. Further Modifying Application Creation

The new-app command generates OpenShift Online objects that will build, deploy, and run the application being created. Normally, these objects are created in the current project using names derived from the input source repositories or the input images. However, new-app allows you to modify this behavior.

The set of objects created by new-app depends on the artifacts passed as input: source repositories, images, or templates.

Table 2.2. new-app Output Objects

ObjectDescription

BuildConfig

A BuildConfig is created for each source repository specified in the command line. The BuildConfig specifies the strategy to use, the source location, and the build output location.

ImageStreams

For BuildConfig, two ImageStreams are usually created. One represents the input image. With Source builds, this is the builder image. The second one represents the output image. If a container image was specified as input to new-app, then an image stream is created for that image as well.

DeploymentConfig

A DeploymentConfig is created either to deploy the output of a build, or a specified image. The new-app command creates emptyDir volumes for all Docker volumes that are specified in containers included in the resulting DeploymentConfig.

Service

The new-app command attempts to detect exposed ports in input images. It uses the lowest numeric exposed port to generate a service that exposes that port. In order to expose a different port, after new-app has completed, simply use the oc expose command to generate additional services.

Other

Other objects may be generated when instantiating templates, according to the template.

2.2.2.4.1. Specifying Environment Variables

When generating applications from a template, source, or an image, you can use the -e|--env argument to pass environment variables to the application container at run time:

$ oc new-app openshift/postgresql-92-centos7 \
    -e POSTGRESQL_USER=user \
    -e POSTGRESQL_DATABASE=db \
    -e POSTGRESQL_PASSWORD=password

The variables can also be read from file using the --env-file argument:

$ cat postgresql.env
POSTGRESQL_USER=user
POSTGRESQL_DATABASE=db
POSTGRESQL_PASSWORD=password
$ oc new-app openshift/postgresql-92-centos7 --env-file=postgresql.env

Additionally, environment variables can be given on standard input by using --env-file=-:

$ cat postgresql.env | oc new-app openshift/postgresql-92-centos7 --env-file=-

See Managing Environment Variables for more information.

Note

Any BuildConfig objects created as part of new-app processing will not be updated with environment variables passed via the -e|--env or --env-file argument.

2.2.2.4.2. Specifying Build Environment Variables

When generating applications from a template, source, or an image, you can use the --build-env argument to pass environment variables to the build container at run time:

$ oc new-app openshift/ruby-23-centos7 \
    --build-env HTTP_PROXY=http://myproxy.net:1337/ \
    --build-env GEM_HOME=~/.gem

The variables can also be read from a file using the --build-env-file argument:

$ cat ruby.env
HTTP_PROXY=http://myproxy.net:1337/
GEM_HOME=~/.gem
$ oc new-app openshift/ruby-23-centos7 --build-env-file=ruby.env

Additionally, environment variables can be given on standard input by using --build-env-file=-:

$ cat ruby.env | oc new-app openshift/ruby-23-centos7 --build-env-file=-
2.2.2.4.3. Specifying Labels

When generating applications from source, images, or templates, you can use the -l|--label argument to add labels to the created objects. Labels make it easy to collectively select, configure, and delete objects associated with the application.

$ oc new-app https://github.com/openshift/ruby-hello-world -l name=hello-world
2.2.2.4.4. Viewing the Output Without Creation

To see a dry-run of what new-app will create, you can use the -o|--output argument with a yaml or json value. You can then use the output to preview the objects that will be created, or redirect it to a file that you can edit. Once you are satisfied, you can use oc create to create the OpenShift Online objects.

To output new-app artifacts to a file, edit them, then create them:

$ oc new-app https://github.com/openshift/ruby-hello-world \
    -o yaml > myapp.yaml
$ vi myapp.yaml
$ oc create -f myapp.yaml
2.2.2.4.5. Creating Objects With Different Names

Objects created by new-app are normally named after the source repository, or the image used to generate them. You can set the name of the objects produced by adding a --name flag to the command:

$ oc new-app https://github.com/openshift/ruby-hello-world --name=myapp
2.2.2.4.6. Creating Objects in a Different Project

Normally, new-app creates objects in the current project. However, you can create objects in a different project by using the -n|--namespace argument:

$ oc new-app https://github.com/openshift/ruby-hello-world -n myproject
2.2.2.4.7. Creating Multiple Objects

The new-app command allows creating multiple applications specifying multiple parameters to new-app. Labels specified in the command line apply to all objects created by the single command. Environment variables apply to all components created from source or images.

To create an application from a source repository and a Docker Hub image:

$ oc new-app https://github.com/openshift/ruby-hello-world mysql
Note

If a source code repository and a builder image are specified as separate arguments, new-app uses the builder image as the builder for the source code repository. If this is not the intent, specify the required builder image for the source using the ~ separator.

2.2.2.4.8. Grouping Images and Source in a Single Pod

The new-app command allows deploying multiple images together in a single pod. In order to specify which images to group together, use the + separator. The --group command line argument can also be used to specify the images that should be grouped together. To group the image built from a source repository with other images, specify its builder image in the group:

$ oc new-app nginx+mysql

To deploy an image built from source and an external image together:

$ oc new-app \
    ruby~https://github.com/openshift/ruby-hello-world \
    mysql \
    --group=ruby+mysql

2.2.3. Creating an Application Using the Web Console

  1. While in the desired project, click Add to Project:

    Web Console Create
  2. Select either a builder image from the list of images in your project, or from the global library:

    Select Builder Image
    Note

    Only image stream tags that have the builder tag listed in their annotations appear in this list, as demonstrated here:

    kind: "ImageStream"
    apiVersion: "v1"
    metadata:
      name: "ruby"
      creationTimestamp: null
    spec:
      dockerImageRepository: "registry.access.redhat.com/openshift3/ruby-20-rhel7"
      tags:
        -
          name: "2.0"
          annotations:
            description: "Build and run Ruby 2.0 applications"
            iconClass: "icon-ruby"
            tags: "builder,ruby" 1
            supports: "ruby:2.0,ruby"
            version: "2.0"
    1
    Including builder here ensures this ImageStreamTag appears in the web console as a builder.
  3. Modify the settings in the new application screen to configure the objects to support your application:

    Create from source
    The builder image name and description.
    The application name used for the generated OpenShift Online objects.
    The Git repository URL, reference, and context directory for your source code.
    Routing configuration section for making this application publicly accessible.
    Build configuration section for customizing build triggers.
    Deployment configuration section for customizing deployment triggers and image environment variables.
    Replica scaling section for configuring the number of running instances of the application.
    The labels to assign to all items generated for the application. You can add and edit labels for all objects here.
    Note

    To see all of the configuration options, click the "Show advanced build and deployment options" link.

2.3. Promoting Applications Across Environments

2.3.1. Overview

Application promotion means moving an application through various runtime environments, typically with an increasing level of maturity. For example, an application might start out in a development environment, then be promoted to a stage environment for further testing, before finally being promoted into a production environment. As changes are introduced in the application, again the changes will start in development and be promoted through stage and production.

The "application" today is more than just the source code written in Java, Perl, Python, etc. It is more now than the static web content, the integration scripts, or the associated configuration for the language specific runtimes for the application. It is more than the application specific archives consumed by those language specific runtimes.

In the context of OpenShift Online and its combined foundation of Kubernetes and Docker, additional application artifacts include:

  • Docker container images with their rich set of metadata and associated tooling.
  • Environment variables that are injected into containers for application use.
  • API objects (also known as resource definitions; see Core Concepts) of OpenShift Online, which:

    • are injected into containers for application use.
    • dictate how OpenShift Online manages containers and pods.

In examining how to promote applications in OpenShift Online, this topic will:

  • Elaborate on these new artifacts introduced to the application definition.
  • Describe how you can demarcate the different environments for your application promotion pipeline.
  • Discuss methodologies and tools for managing these new artifacts.
  • Provide examples that apply the various concepts, constructs, methodologies, and tools to application promotion.

2.3.2. Application Components

2.3.2.1. API Objects

With regard to OpenShift Online and Kubernetes resource definitions (the items newly introduced to the application inventory), there are a couple of key design points for these API objects that are relevant to revisit when considering the topic of application promotion.

First, as highlighted throughout OpenShift Online documentation, every API object can be expressed via either JSON or YAML, making it easy to manage these resource definitions via traditional source control and scripting.

Also, the API objects are designed such that there are portions of the object which specify the desired state of the system, and other portions which reflect the status or current state of the system. This can be thought of as inputs and outputs. The input portions, when expressed in JSON or YAML, in particular are items that fit naturally as source control managed (SCM) artifacts.

Note

Remember, the input or specification portions of the API objects can be totally static or dynamic in the sense that variable substitution via template processing is possible on instantiation.

The result of these points with respect to API objects is that with their expression as JSON or YAML files, you can treat the configuration of the application as code.

Conceivably, almost any of the API objects may be considered an application artifact by your organization. Listed below are the objects most commonly associated with deploying and managing an application:

BuildConfigs
This is a special case resource in the context of application promotion. While a BuildConfig is certainly a part of the application, especially from a developer’s perspective, typically the BuildConfig is not promoted through the pipeline. It produces the Image that is promoted (along with other items) through the pipeline.
Templates
In terms of application promotion, Templates can serve as the starting point for setting up resources in a given staging environment, especially with the parameterization capabilities. Additional post-instantiation modifications are very conceivable though when applications move through a promotion pipeline. See Scenarios and Examples for more on this.
Routes
These are the most typical resources that differ stage to stage in the application promotion pipeline, as tests against different stages of an application access that application via its Route. Also, remember that you have options with regard to manual specification or auto-generation of host names, as well as the HTTP-level security of the Route.
Services
If reasons exist to avoid Routers and Routes at given application promotion stages (perhaps for simplicity’s sake for individual developers at early stages), an application can be accessed via the Cluster IP address and port. If used, some management of the address and port between stages could be warranted.
Endpoints
Certain application-level services (e.g., database instances in many enterprises) may not be managed by OpenShift Online. If so, then creating those Endpoints yourself, along with the necessary modifications to the associated Service (omitting the selector field on the Service) are activities that are either duplicated or shared between stages (based on how you delineate your environment).
Secrets
The sensitive information encapsulated by Secrets are shared between staging environments when the corresponding entity (either a Service managed by OpenShift Online or an external service managed outside of OpenShift Online) the information pertains to is shared. If there are different versions of the said entity in different stages of your application promotion pipeline, it may be necessary to maintain a distinct Secret in each stage of the pipeline or to make modifications to it as it traverses through the pipeline. Also, take care that if you are storing the Secret as JSON or YAML in an SCM, some form of encryption to protect the sensitive information may be warranted.
DeploymentConfigs
This object is the primary resource for defining and scoping the environment for a given application promotion pipeline stage; it controls how your application starts up. While there are aspects of it that will be common across all the different stage, undoubtedly there will be modifications to this object as it progresses through your application promotion pipeline to reflect differences in the environments for each stage, or changes in behavior of the system to facilitate testing of the different scenarios your application must support.
ImageStreams, ImageStreamTags, and ImageStreamImage
Detailed in the Images and Image Streams sections, these objects are central to the OpenShift Online additions around managing container images.
ServiceAccounts and RoleBindings
Management of permissions to other API objects within OpenShift Online, as well as the external services, are intrinsic to managing your application. Similar to Secrets, the ServiceAccounts and RoleBindingscan objects vary in how they are shared between the different stages of your application promotion pipeline based on your needs to share or isolate those different environments.
PersistentVolumeClaims
Relevant to stateful services like databases, how much these are shared between your different application promotion stages directly correlates to how your organization shares or isolates the copies of your application data.
ConfigMaps
A useful decoupling of Pod configuration from the Pod itself (think of an environment variable style configuration), these can either be shared by the various staging environments when consistent Pod behavior is desired. They can also be modified between stages to alter Pod behavior (usually as different aspects of the application are vetted at different stages).

2.3.2.2. Images

As noted earlier, container images are now artifacts of your application. In fact, of the new applications artifacts, images and the management of images are the key pieces with respect to application promotion. In some cases, an image might encapsulate the entirety of your application, and the application promotion flow consists solely of managing the image.

Images are not typically managed in a SCM system, just as application binaries were not in previous systems. However, just as with binaries, installable artifacts and corresponding repositories (that is, RPMs, RPM repositories, Nexus, etc.) arose with similar semantics to SCMs, similar constructs and terminology around image management that are similar to SCMs have arisen:

  • Image registry == SCM server
  • Image repository == SCM repository

As images reside in registries, application promotion is concerned with ensuring the appropriate image exists in a registry that can be accessed from the environment that needs to run the application represented by that image.

Rather than reference images directly, application definitions typically abstract the reference into an image stream. This means the image stream will be another API object that makes up the application components. For more details on image streams, see Core Concepts.

2.3.2.3. Summary

Now that the application artifacts of note, images and API objects, have been detailed in the context of application promotion within OpenShift Online, the notion of where you run your application in the various stages of your promotion pipeline is next the point of discussion.

2.3.3. Deployment Environments

A deployment environment, in this context, describes a distinct space for an application to run during a particular stage of a CI/CD pipeline. Typical environments include development, test, stage, and production, for example. The boundaries of an environment can be defined in different ways, such as:

  • Via labels and unique naming within a single project.
  • Via distinct projects within a cluster.
  • Via distinct clusters.

And it is conceivable that your organization leverages all three.

2.3.3.1. Considerations

Typically, you will consider the following heuristics in how you structure the deployment environments:

  • How much resource sharing the various stages of your promotion flow allow
  • How much isolation the various stages of your promotion flow require
  • How centrally located (or geographically dispersed) the various stages of your promotion flow are

Also, some important reminders on how OpenShift Online clusters and projects relate to image registries:

  • Multiple project in the same cluster can access the same image streams.
  • Multiple clusters can access the same external registries.
  • Clusters can only share a registry if the OpenShift Online internal image registry is exposed via a route.

2.3.3.2. Summary

After deployment environments are defined, promotion flows with delineation of stages within a pipeline can be implemented. The methods and tools for constructing those promotion flow implementations are the next point of discussion.

2.3.4. Methods and Tools

Fundamentally, application promotion is a process of moving the aforementioned application components from one environment to another. The following subsections outline tools that can be used to move the various components by hand, before advancing to discuss holistic solutions for automating application promotion.

Note

There are a number of insertion points available during both the build and deployment processes. They are defined within BuildConfig and DeploymentConfig API objects. These hooks allow for the invocation of custom scripts which can interact with deployed components such as databases, and with the OpenShift Online cluster itself.

Therefore, it is possible to use these hooks to perform component management operations that effectively move applications between environments, for example by performing an image tag operation from within a hook. However, the various hook points are best suited to managing an application’s lifecycle within a given environment (for example, using them to perform database schema migrations when a new version of the application is deployed), rather than to move application components between environments.

2.3.4.1. Managing API Objects

Resources, as defined in one environment, will be exported as JSON or YAML file content in preparation for importing it into a new environment. Therefore, the expression of API objects as JSON or YAML serves as the unit of work as you promote API objects through your application pipeline. The oc CLI is used to export and import this content.

Tip

While not required for promotion flows with OpenShift Online, with the JSON or YAML stored in files, you can consider storing and retrieving the content from a SCM system. This allows you to leverage the versioning related capabilities of the SCM, including the creation of branches, and the assignment of and query on various labels or tags associated to versions.

2.3.4.1.1. Exporting API Object State

API object specifications should be captured with oc export. This operation removes environment specific data from the object definitions (e.g., current namespace or assigned IP addresses), allowing them to be recreated in different environments (unlike oc get operations, which output an unfiltered state of the object).

Use of oc label, which allows for adding, modifying, or removing labels on API objects, can prove useful as you organize the set of object collected for promotion flows, because labels allow for selection and management of groups of pods in a single operation. This makes it easier to export the correct set of objects and, because the labels will carry forward when the objects are created in a new environment, they also make for easier management of the application components in each environment.

Note

API objects often contain references such as a DeploymentConfig that references a Secret. When moving an API object from one environment to another, you must ensure that such references are also moved to the new environment.

Similarly, API objects such as a DeploymentConfig often contain references to ImageStreams that reference an external registry. When moving an API object from one environment to another, you must ensure such references are resolvable within the new environment, meaning that the reference must be resolvable and the ImageStream must reference an accessible registry in the new environment. See Moving Images and Promotion Caveats for more detail.

2.3.4.1.2. Importing API Object State
2.3.4.1.2.1. Initial Creation

The first time an application is being introduced into a new environment, it is sufficient to take the JSON or YAML expressing the specifications of your API objects and run oc create to create them in the appropriate environment. When using oc create, keep the --save-config option in mind. Saving configuration elements on the object in its annotation list facilitates the later use of oc apply to modify the object.

2.3.4.1.2.2. Iterative Modification

After the various staging environments are initially established, as promotion cycles commence and the application moves from stage to stage, the updates to your application can include modification of the API objects that are part of the application. Changes in these API objects are conceivable since they represent the configuration for the OpenShift Online system. Motivations for such changes include:

  • Accounting for environmental differences between staging environments.
  • Verifying various scenarios your application supports.

Transfer of the API objects to the next stage’s environment is accomplished via use of the oc CLI. While a rich set of oc commands which modify API objects exist, this topic focuses on oc apply, which computes and applies differences between objects.

Specifically, you can view oc apply as a three-way merge that takes in files or stdin as the input along with an existing object definition. It performs a three-way merge between:

  1. the input into the command,
  2. the current version of the object, and
  3. the most recent user specified object definition stored as an annotation in the current object.

The existing object is then updated with the result.

If further customization of the API objects is necessary, as in the case when the objects are not expected to be identical between the source and target environments, oc commands such as oc set can be used to modify the object after applying the latest object definitions from the upstream environment.

Some specific usages are cited in Scenarios and Examples.

2.3.4.2. Managing Images and Image Streams

Images in OpenShift Online are managed via a series of API objects as well. However, managing images are so central to application promotion that discussion of the tools and API objects most directly tied to images warrant separate discussion. Both manual and automated forms exist to assist you in managing image promotion (the propagation of images through your pipeline).

2.3.4.2.1. Moving Images
Note

For all the detailed caveats around managing images, refer to the Managing Images topic.

2.3.4.2.1.1. When Staging Environments Share a Registry

When your staging environments share the same OpenShift Online registry, for example if they are all on the same OpenShift Online cluster, there are two operations that are the basic means of moving your images between the stages of your application promotion pipeline:

  1. First, analogous to docker tag and git tag, the oc tag command allows you to update an OpenShift Online image stream with a reference to a specific image. It also allows you to copy references to specific versions of an image from one image stream to another, even across different projects in a cluster.
  2. Second, the oc import-image serves as a bridge between external registries and image streams. It imports the metadata for a given image from the registry and stores it into the image stream as an image stream tag. Various BuildConfigs and DeploymentConfigs in your project can reference those specific images.
2.3.4.2.1.2. When Staging Environments Use Different Registries

More advanced usage occurs when your staging environments leverage different OpenShift Online registries. Accessing the Internal Registry spells out the steps in detail, but in summary you can:

  1. Use the docker command in conjunction which obtaining the OpenShift Online access token to supply into your docker login command.
  2. After being logged into the OpenShift Online registry, use docker pull, docker tag and docker push to transfer the image.
  3. After the image is available in the registry of the next environment of your pipeline, use oc tag as needed to populate any image streams.
2.3.4.2.2. Deploying

Whether changing the underlying application image or the API objects that configure the application, a deployment is typically necessary to pick up the promoted changes. If the images for your application change (for example, due to an oc tag operation or a docker push as part of promoting an image from an upstream environment), ImageChangeTriggers on your DeploymentConfig can trigger the new deployment. Similarly, if the DeploymentConfig API object itself is being changed, a ConfigChangeTrigger can initiate a deployment when the API object is updated by the promotion step (for example, oc apply).

Otherwise, the oc commands that facilitate manual deployment include:

  • oc deploy: The original method to view, start, cancel, or retry deployments.
  • oc rollout: The new approach to manage deployments, including pause and resume semantics and richer features around managing history.
  • oc rollback: Allows for reversion to a previous deployment; in the promotion scenario, if testing of a new version encounters issues, confirming it still works with the previous version could be warranted.
2.3.4.2.3. Automating Promotion Flows with Jenkins

After you understand the components of your application that need to be moved between environments when promoting it and the steps required to move the components, you can start to orchestrate and automate the workflow. OpenShift Online provides a Jenkins image and plug-ins to help with this process.

The OpenShift Online Jenkins image is detailed in Using Images, including the set of OpenShift Online-centric plug-ins that facilitate the integration of Jenkins, and Jenkins Pipelines. Also, the Pipeline build strategy facilitates the integration between Jenkins Pipelines and OpenShift Online. All of these focus on enabling various aspects of CI/CD, including application promotion.

When moving beyond manual execution of application promotion steps, the Jenkins-related features provided by OpenShift Online should be kept in mind:

  • OpenShift Online provides a Jenkins image that is heavily customized to greatly ease deployment in an OpenShift Online cluster.
  • The Jenkins image contains the OpenShift Pipeline plug-in, which provides building blocks for implementing promotion workflows. These building blocks include the triggering of Jenkins jobs as image streams change, as well as the triggering of builds and deployments within those jobs.
  • BuildConfigs employing the OpenShift Online Jenkins Pipeline build strategy enable execution of Jenkinsfile-based Jenkins Pipeline jobs. Pipeline jobs are the strategic direction within Jenkins for complex promotion flows and can leverage the steps provided by the OpenShift Pipeline Plug-in.
2.3.4.2.4. Promotion Caveats
2.3.4.2.4.1. API Object References

API objects can reference other objects. A common use for this is to have a DeploymentConfig that references an image stream, but other reference relationships may also exist.

When copying an API object from one environment to another, it is critical that all references can still be resolved in the target environment. There are a few reference scenarios to consider:

  • The reference is "local" to the project. In this case, the referenced object resides in the same project as the object that references it. Typically the correct thing to do is to ensure that you copy the referenced object into the target environment in the same project as the object referencing it.
  • The reference is to an object in another project. This is typical when an image stream in a shared project is used by multiple application projects (see Managing Images). In this case, when copying the referencing object to the new environment, you must update the reference as needed so it can be resolved in the target environment. That may mean:

    • Changing the project the reference points to, if the shared project has a different name in the target environment.
    • Moving the referenced object from the shared project into the local project in the target environment and updating the reference to point to the local project when moving the primary object into the target environment.
    • Some other combination of copying the referenced object into the target environment and updating references to it.

In general, the guidance is to consider objects referenced by the objects being copied to a new environment and ensure the references are resolvable in the target environment. If not, take appropriate action to fix the references and make the referenced objects available in the target environment.

2.3.4.2.4.2. Image Registry References

Image streams point to image repositories to indicate the source of the image they represent. When an image stream is moved from one environment to another, it is important to consider whether the registry and repository reference should also change:

  • If different image registries are used to assert isolation between a test environment and a production environment.
  • If different image repositories are used to separate test and production-ready images.

If either of these are the case, the image stream must be modified when it is copied from the source environment to the target environment so that it resolves to the correct image. This is in addition to performing the steps described in Scenarios and Examples to copy the image from one registry and repository to another.

2.3.4.3. Summary

At this point, the following have been defined:

  • New application artifacts that make up a deployed application.
  • Correlation of application promotion activities to tools and concepts provided by OpenShift Online.
  • Integration between OpenShift Online and the CI/CD pipeline engine Jenkins.

Putting together examples of application promotion flows within OpenShift Online is the final step for this topic.

2.3.5. Scenarios and Examples

Having defined the new application artifact components introduced by the Docker, Kubernetes, and OpenShift Online ecosystems, this section covers how to promote those components between environments using the mechanisms and tools provided by OpenShift Online.

Of the components making up an application, the image is the primary artifact of note. Taking that premise and extending it to application promotion, the core, fundamental application promotion pattern is image promotion, where the unit of work is the image. The vast majority of application promotion scenarios entails management and propagation of the image through the promotion pipeline.

Simpler scenarios solely deal with managing and propagating the image through the pipeline. As the promotion scenarios broaden in scope, the other application artifacts, most notably the API objects, are included in the inventory of items managed and propagated through the pipeline.

This topic lays out some specific examples around promoting images as well as API objects, using both manual and automated approaches. But first, note the following on setting up the environment(s) for your application promotion pipeline.

2.3.5.1. Setting up for Promotion

After you have completed development of the initial revision of your application, the next logical step is to package up the contents of the application so that you can transfer to the subsequent staging environments of your promotion pipeline.

  1. First, group all the API objects you view as transferable and apply a common label to them:

    labels:
      promotion-group: <application_name>

    As previously described, the oc label command facilitates the management of labels with your various API objects.

    Tip

    If you initially define your API objects in a OpenShift Online template, you can easily ensure all related objects have the common label you will use to query on when exporting in preparation for a promotion.

  2. You can leverage that label on subsequent queries. For example, consider the following set of oc command invocations that would then achieve the transfer of your application’s API objects:

    $ oc login <source_environment>
    $ oc project <source_project>
    $ oc export dc,is,svc,route,secret,sa -l promotion-group=<application_name> -o yaml > export.yaml
    $ oc login <target_environment>
    $ oc new-project <target_project> 1
    $ oc create -f export.yaml
    1
    Alternatively, oc project <target_project> if it already exists.
    Note

    On the oc export command, whether or not you include the is type for image streams depends on how you choose to manage images, image streams, and registries across the different environments in your pipeline. The caveats around this are discussed below. See also the Managing Images topic.

  3. You must also get any tokens necessary to operate against each registry used in the different staging environments in your promotion pipeline. For each environment:

    1. Log in to the environment:

      $ oc login <each_environment_with_a_unique_registry>
    2. Get the access token with:

      $ oc whoami -t
    3. Copy and paste the token value for later use.

2.3.5.2. Repeatable Promotion Process

After the initial setup of the different staging environments for your pipeline, a set of repeatable steps to validate each iteration of your application through the promotion pipeline can commence. These basic steps are taken each time the image or API objects in the source environment are changed:

Move updated images → Move updated API objects → Apply environment specific customizations

  1. Typically, the first step is promoting any updates to the image(s) associated with your application to the next stage in the pipeline. As noted above, the key differentiator in promoting images is whether the OpenShift Online registry is shared or not between staging environments.

    1. If the registry is shared, simply leverage oc tag:

      $ oc tag <project_for_stage_N>/<imagestream_name_for_stage_N>:<tag_for_stage_N> <project_for_stage_N+1>/<imagestream_name_for_stage_N+1>:<tag_for_stage_N+1>
    2. If the registry is not shared, you can leverage the access tokens for each of your promotion pipeline registries as you log into both the source and destination registries, pulling, tagging, and pushing your application images accordingly:

      1. Log in to the source environment registry:

        $ docker login -u <username> -e <any_email_address> -p <token_value> <src_env_registry_ip>:<port>
      2. Pull your application’s image:

        $ docker pull <src_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
      3. Tag your application’s image to the destination registry’s location, updating namespace, name, and tag as needed to conform to the destination staging environment:

        $ docker tag <src_env_registry_ip>:<port>/<namespace>/<image name>:<tag> <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
      4. Log into the destination staging environment registry:

        $ docker login -u <username> -e <any_email_address> -p <token_value> <dest_env_registry_ip>:<port>
      5. Push the image to its destination:

        $ docker push <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
        Tip

        To automatically import new versions of an image from an external registry, the oc tag command has a --scheduled option. If used, the image the ImageStreamTag references will be periodically pulled from the registry hosting the image.

  2. Next, there are the cases where the evolution of your application necessitates fundamental changes to your API objects or additions and deletions from the set of API objects that make up the application. When such evolution in your application’s API objects occurs, the OpenShift Online CLI provides a broad range of options to transfer to changes from one staging environment to the next.

    1. Start in the same fashion as you did when you initially set up your promotion pipeline:

      $ oc login <source_environment>
      $ oc project <source_project>
      $ oc export dc,is,svc,route,secret,sa -l template=<application_template_name> -o yaml > export.yaml
      $ oc login <target_environment>
      $ oc <target_project>
    2. Rather than simply creating the resources in the new environment, update them. You can do this a few different ways:

      1. The more conservative approach is to leverage oc apply and merge the new changes to each API object in the target environment. In doing so, you can --dry-run=true option and examine the resulting objects prior to actually changing the objects:

        $ oc apply -f export.yaml --dry-run=true

        If satisfied, actually run the apply command:

        $ oc apply -f export.yaml

        The apply command optionally takes additional arguments that help with more complicated scenarios. See oc apply --help for more details.

      2. Alternatively, the simpler but more aggressive approach is to leverage oc replace. There is no dry run with this update and replace. In the most basic form, this involves executing:

        $ oc replace -f export.yaml

        As with apply, replace optionally takes additional arguments for more sophisticated behavior. See oc replace --help for more details.

  3. The previous steps automatically handle new API objects that were introduced, but if API objects were deleted from the source environment, they must be manually deleted from the target environment using oc delete.
  4. Tuning of the environment variables cited on any of the API objects may be necessary as the desired values for those may differ between staging environments. For this, use oc set env:

    $ oc set env <api_object_type>/<api_object_ID> <env_var_name>=<env_var_value>
  5. Finally, trigger a new deployment of the updated application using the oc rollout command or one of the other mechanisms discussed in the Deployments section above.

2.3.5.3. Repeatable Promotion Process Using Jenkins

The OpenShift Sample job defined in the Jenkins Docker Image for OpenShift Online is an example of image promotion within OpenShift Online within the constructs of Jenkins. Setup for this sample is located in the OpenShift Origin source repository.

This sample includes:

  • Use of Jenkins as the CI/CD engine.
  • Use of the OpenShift Pipeline plug-in for Jenkins. This plug-in provides a subset of the functionality provided by the oc CLI for OpenShift Online packaged as Jenkins Freestyle and DSL Job steps. Note that the oc binary is also included in the Jenkins Docker Image for OpenShift Online, and can also be used to interact with OpenShift Online in Jenkins jobs.
  • The OpenShift Online-provided templates for Jenkins. There is a template for both ephemeral and persistent storage.
  • A sample application: defined in the OpenShift Origin source repository, this application leverages ImageStreams, ImageChangeTriggers, ImageStreamTags, BuildConfigs, and separate DeploymentConfigs and Services corresponding to different stages in the promotion pipeline.

The following examines the various pieces of the OpenShift Sample job in more detail:

  1. The first step is the equivalent of an oc scale dc frontend --replicas=0 call. This step is intended to bring down any previous versions of the application image that may be running.
  2. The second step is the equivalent of an oc start-build frontend call.
  3. The third step is the equivalent of an oc deploy frontend --latest or oc rollout latest dc/frontend call.
  4. The fourth step is the "test" for this sample. It ensures that the associated service for this application is in fact accessible from a network perspective. Under the covers, a socket connection is attempted against the IP address and port associated with the OpenShift Online service. Of course, additional tests can be added (if not via OpenShift Pipepline plug-in steps, then via use of the Jenkins Shell step to leverage OS-level commands and scripts to test your application).
  5. The fifth step commences under that assumption that the testing of your application passed and hence intends to mark the image as "ready". In this step, a new prod tag is created for the application image off of the latest image. With the frontend DeploymentConfig having an ImageChangeTriggerdefined for that tag, the corresponding "production" deployment is launched.
  6. The sixth and last step is a verification step, where the plug-in confirms that OpenShift Online launched the desired number of replicas for the "production" deployment.

Chapter 3. Authentication

3.1. Web Console Authentication

When accessing the web console from a browser, you are automatically redirected to a login page.

Review the browser versions and operating systems that can be used to access the web console.

You can provide your login credentials on this page to obtain a token to make API calls. After logging in, you can navigate your projects using the web console.

3.2. CLI Authentication

You can authenticate from the command line using the CLI command oc login. You can get started with the CLI by running this command

with the url of the online cluster you are using:

$ oc login https://<online_cluster_url>

The command’s interactive flow helps you establish a session to an OpenShift Online server with the provided credentials. If any information required to successfully log in to an OpenShift Online server is not provided, the command prompts for user input as required. The configuration is automatically saved and is then used for every subsequent command.

All configuration options for the oc login command, listed in the oc login --help command output, are optional. The following example shows usage with some common options:

$ oc login [-u=<username>] \
  [-p=<password>] \
  [-s=<server>] \
  [-n=<project>] \
  [--certificate-authority=</path/to/file.crt>|--insecure-skip-tls-verify]

The following table describes these common options:

Table 3.1. Common CLI Configuration Options

OptionSyntaxDescription

-s, --server

$ oc login -s=<server>

Specifies the host name of the OpenShift Online server. If a server is provided through this flag, the command does not ask for it interactively. This flag can also be used if you already have a CLI configuration file and want to log in and switch to another server.

-u, --username and -p, --password

$ oc login -u=<username> -p=<password>

Allows you to specify the credentials to log in to the OpenShift Online server. If user name or password are provided through these flags, the command does not ask for it interactively. These flags can also be used if you already have a configuration file with a session token established and want to log in and switch to another user name.

-n, --namespace

$ oc login -u=<username> -p=<password> -n=<project>

A global CLI option which, when used with oc login, allows you to specify the project to switch to when logging in as a given user.

--certificate-authority

$ oc login --certificate-authority=<path/to/file.crt>

Correctly and securely authenticates with an OpenShift Online server that uses HTTPS. The path to a certificate authority file must be provided.

--insecure-skip-tls-verify

$ oc login --insecure-skip-tls-verify

Allows interaction with an HTTPS server bypassing the server certificate checks; however, note that it is not secure. If you try to oc login to a HTTPS server that does not provide a valid certificate, and this or the --certificate-authority flags were not provided, oc login will prompt for user input to confirm (y/N kind of input) about connecting insecurely.

CLI configuration files allow you to easily manage multiple CLI profiles.

Chapter 4. Authorization

4.1. Overview

This topic contains authorization tasks for application developers and their capabilities, as dictated by the cluster administrator.

4.2. Checking If Users Can Create Pods

Using the scc-review and scc-subject-review options, you can see if an individual user, or a user under a specific service account, can create or update a pod.

Using the scc-review option, you can check if a service account can create or update a pod. The command outputs the security context constraints that admit the resource.

For example, to check if a user with the system:serviceaccount:projectname:default service account can a create a pod:

$ oc policy scc-review -z system:serviceaccount:projectname:default -f my_resource.yaml

You can also use the scc-subject-review option to check whether a specific user can create or update a pod:

$ oc policy scc-subject-review -u <username> -f my_resource.yaml

To check if a user belonging to a specific group can create a pod in a specific file:

$ oc policy scc-subject-review -u <username> -g <groupname> -f my_resource.yaml

4.3. Determining What You Can Do as an Authenticated User

From within your OpenShift Online project, you can determine what verbs you can perform against all namespace-scoped resources (including third-party resources).

The can-i command option tests scopes in terms of the user and role.

$ oc policy can-i --list --loglevel=8

The output helps you to determine what API request to make to gather the information.

To receive information back in a user-readable format, run:

$ oc policy can-i --list

The output provides a full list.

To determine if you can perform specific verbs, run:

$ oc policy can-i <verb> <resource>

User scopes can provide more information about a given scope. For example:

$ oc policy can-i <verb> <resource> --scopes=user:info

Chapter 5. Projects

5.1. Overview

A project allows a community of users to organize and manage their content in isolation from other communities.

5.2. Creating a Project

If allowed , you can create a new project using the CLI or the web console.

To create a new project using the CLI:

$ oc new-project <project_name> \
    --description="<description>" --display-name="<display_name>"

For example:

$ oc new-project hello-openshift \
    --description="This is an example project to demonstrate OpenShift v3" \
    --display-name="Hello OpenShift"
Note

The number of projects you are allowed to create is limited. Once your limit is reached, you may need to delete an existing project in order to create a new one.

5.3. Viewing Projects

When viewing projects, you are restricted to seeing only the projects you have access to view based on the authorization policy.

To view a list of projects:

$ oc get projects

You can change from the current project to a different project for CLI operations. The specified project is then used in all subsequent operations that manipulate project-scoped content:

$ oc project <project_name>

You can also use the web console to view and change between projects. After authenticating and logging in, you are presented with a list of projects that you have access to:

Projects Page

If you use the CLI to create a new project, you can then refresh the page in the browser to see the new project.

Selecting a project brings you to the project overview for that project.

5.4. Checking Project Status

The oc status command provides a high-level overview of the current project, with its components and their relationships. This command takes no argument:

$ oc status

5.5. Filtering by Labels

You can filter the contents of a project page in the web console by using the labels of a resource. You can pick from a suggested label name and values, or type in your own. Multiple filters can be added. When multiple filters are applied, resources must match all of the filters to remain visible.

To filter by labels:

  1. Select a label type:

    Web Console Filter Step 1
  2. Select one of the following:

    exists

    Verify that the label name exists, but ignore its value.

    in

    Verify that the label name exists and is equal to one of the selected values.

    not in

    Verify that the label name does not exist, or is not equal to any of the selected values.

    Web Console Filter Step 2
    1. If you selected in or not in, select a set of values then select Filter:

      Web Console Filter Step 3
  3. After adding filters, you can stop filtering by selecting Clear all filters or by clicking individual filters to remove them:

    Web Console Filter Active

5.6. Deleting a Project

When you delete a project, the server updates the project status to Terminating from Active. The server then clears all content from a project that is Terminating before finally removing the project. While a project is in Terminating status, a user cannot add new content to the project. Projects can be deleted from the CLI or the web console.

To delete a project using the CLI:

$ oc delete project <project_name>

5.7. Project Collaboration in OpenShift Online Pro

Important

Collaboration is not available in OpenShift Online Starter.

Every OpenShift Online Pro account has the ability to add up to 50 collaborator users per subscription. These collaborator users are granted cluster access from OpenShift Online Pro account subscribers so that they can collaborate on projects hosted with OpenShift Online. This allows multiple users to gain access to projects under a single subscription, without having to pay a monthly fee for every account.

Note

Existing OpenShift Online Pro users do not need to be added as collaborators.

5.7.1. Collaboration Restrictions

Collaborators can only access the resources within the projects that they have been granted access. Also, their ability to view, edit, and manage the project resources will depend on the specific role that they have been granted within the project.

5.7.2. Adding Collaborators

OpenShift Online Pro subscribers can add collaborators by following these steps:

  1. Each user you want to add as a collaborator must create a free account at developers.redhat.com. Once your collaborator has confirmed their Red Hat Developers account, you can add them to your subscription.
  2. Each collaborator must sign into developers.redhat.com and click on their name in the upper-right corner to access their account details. Make note of the Red Hat Login ID on this page, as it is the user name you will be required to enter in order to associate the collaborator with your subscription.
  3. Sign in to manage.openshift.com and click on Manage Subscription under the cluster where you want to add the collaborator.

    manage subscriptions collaboration
  4. Once you are in the subscription management console, click the Manage link under the Collaborators heading, which brings you to the Collaboration page.

    manage collaborators
  5. On the Collaboration page, enter the Red Hat Login ID for the user in the user name field and click Add Collaborator.

    You should now see the user listed under your collaborators, the time the user was added, and an option to remove the user from your subscription.

    This does not automatically grant the user any access to your projects. Access must be granted manually by the project owner using oc policy commands or the web console.

5.7.3. Granting Project Access Using the Web Console

After adding the collaborator to your OpenShift Online Pro subscription, you can grant project access to the collaborator using the web console.

  1. From within a project, click Resources, then Membership.
  2. Add roles (for example, view, edit, or admin) to the user you want to grant access.

    See Authorization for more information on access roles.

    Now, when the collaborator user signs in to manage.openshift.com, they will see a card to log in to the web console for the same cluster as your subscription and, if the collaborator was granted access to your projects on the cluster, they will have access just like any other user.

5.7.4. Granting Project Access Using the CLI

After adding the collaborator to your OpenShift Online Pro subscription, you can grant project access to the collaborator using the CLI.

  1. Log in to the cluster through the CLI using your access token.
  2. Grant the user a role using the same user name listed on the Collaboration page using:

    $ oc policy add-role-to-user <role-name> <username>

    For example:

    ~$ oc login https://api.openshift.com --token=<...>
    Logged into "https://api.openshift.com:443" as "exampleuser" using the token provided.
    
    You have one project on this server: "exampleuser-collab"
    
    Using project "exampleuser-collab".
    ~$ oc policy add-role-to-user view collaborator-1234
    role "view" added: "collaborator-1234"

    This example grants view access to the project for user collaborator-1234. See Authorization for more information on access roles.

    Now, when the collaborator user signs in to manage.openshift.com, they will see a card to log in to the web console for the same cluster as your subscription and, if the collaborator was granted access to your projects on the cluster, they will have access just like any other user.

5.7.5. Removing Collaborators

If at any time you wish to remove the user as a collaborator from your subscription, you can do so on the same Collaboration page you used to add them. It is important to note, however, that this will not automatically remove any roles you have assigned the user in your projects. These will need to be manually deleted, or the user may still have access to your projects.

5.7.5.1. Removing Project Access Using the Web Console

You can remove project access from the collaborator using the web console.

  1. From within a project, click Resources, then Membership.
  2. Remove roles (for example, view, edit, or admin) from the user.

5.7.5.2. Removing Project Access Using the CLI

You can remove project access from the collaborator using the CLI.

  1. Log in to the cluster through the CLI using your access token.
  2. Remove a role (for example, view, edit, or admin) from a specific collaborator using the same user name listed on the Collaboration page by running:

    $ oc policy remove-role-from-user <role-name> <username>

Chapter 6. Migrating Applications

6.1. Overview

This topic covers the migration procedure of OpenShift version 2 (v2) applications to OpenShift version 3 (v3).

To migrate OpenShift v2 applications to OpenShift Online v3, all cartridges in the v2 application must be recorded as each v2 cartridge is equivalent with a corresponding image or template in OpenShift Online v3 and they must be migrated individually. For each cartridge, all dependencies or required packages also must be recorded, as they must be included in the v3 images.

The general migration procedure is:

  1. Back up the v2 application.

    • Web cartridge: The source code can be backed up to a Git repository such as by pushing to a repository on GitHub.
    • Database cartridge: The database can be backed up using a dump command (mongodump, mysqldump, pg_dump) to back up the database.
    • Web and database cartridges: rhc client tool provides snapshot ability to back up multiple cartridges:

      $ rhc snapshot save <app_name>

      The snapshot is a tar file that can be unzipped, and its content is application source code and the database dump.

  2. If the application has a database cartridge, create a v3 database application, sync the database dump to the pod of the new v3 database application, then restore the v2 database in the v3 database application with database restore commands.
  3. For a web framework application, edit the application source code to make it v3 compatible. Then, add any dependencies or packages required in appropriate files in the Git repository. Convert v2 environment variables to corresponding v3 environment variables.
  4. Create a v3 application from source (your Git repository) or from a quickstart with your Git URL. Also, add the database service parameters to the new application to link the database application to the web application.
  5. In v2, there is an integrated Git environment and your applications automatically rebuild and restart whenever a change is pushed to your v2 Git repository. In v3, in order to have a build automatically triggered by source code changes pushed to your public Git repository, you must set up a webhook after the initial build in v3 is completed.

6.2. Migrating Database Applications

6.2.1. Overview

This topic reviews how to migrate MySQL, PostgreSQL, and MongoDB database applications from OpenShift version 2 (v2) to OpenShift version 3 (v3).

6.2.2. Supported Databases

v2v3

MongoDB: 2.4

MongoDB: 2.4, 2.6

MySQL: 5.5

MySQL: 5.5, 5.6

PostgreSQL: 9.2

PostgreSQL: 9.2, 9.4

6.2.3. MySQL

  1. Export all databases to a dump file and copy it to a local machine (into the current directory):

    $ rhc ssh <v2_application_name>
    $ mysqldump --skip-lock-tables -h $OPENSHIFT_MYSQL_DB_HOST -P ${OPENSHIFT_MYSQL_DB_PORT:-3306} -u ${OPENSHIFT_MYSQL_DB_USERNAME:-'admin'} \
     --password="$OPENSHIFT_MYSQL_DB_PASSWORD" --all-databases > ~/app-root/data/all.sql
    $ exit
  2. Download dbdump to your local machine:

    $ mkdir mysqldumpdir
    $ rhc scp -a <v2_application_name> download mysqldumpdir app-root/data/all.sql
  3. Create a v3 mysql-persistent pod from template:

    $ oc new-app mysql-persistent -p \
       MYSQL_USER=<your_V2_mysql_username> -p \
       MYSQL_PASSWORD=<your_v2_mysql_password> -p MYSQL_DATABASE=<your_v2_database_name>
  4. Check to see if the pod is ready to use:

    $ oc get pods
  5. When the pod is up and running, copy database archive files to your v3 MySQL pod:

    $ oc rsync /local/mysqldumpdir <mysql_pod_name>:/var/lib/mysql/data
  6. Restore the database in the v3 running pod:

    $ oc rsh <mysql_pod>
    $ cd /var/lib/mysql/data/mysqldumpdir

    In v3, to restore databases you need to access MySQL as root user.

    In v2, the $OPENSHIFT_MYSQL_DB_USERNAME had full privileges on all databases. In v3, you must grant privileges to $MYSQL_USER for each database.

    $ mysql -u root
    $ source all.sql

    Grant all privileges on <dbname> to <your_v2_username>@localhost, then flush privileges.

  7. Remove the dump directory from the pod:

    $ cd ../; rm -rf /var/lib/mysql/data/mysqldumpdir

Supported MySQL Environment Variables

v2v3

OPENSHIFT_MYSQL_DB_HOST

[service_name]_SERVICE_HOST

OPENSHIFT_MYSQL_DB_PORT

[service_name]_SERVICE_PORT

OPENSHIFT_MYSQL_DB_USERNAME

MYSQL_USER

OPENSHIFT_MYSQL_DB_PASSWORD

MYSQL_PASSWORD

OPENSHIFT_MYSQL_DB_URL

 

OPENSHIFT_MYSQL_DB_LOG_DIR

 

OPENSHIFT_MYSQL_VERSION

 

OPENSHIFT_MYSQL_DIR

 

OPENSHIFT_MYSQL_DB_SOCKET

 

OPENSHIFT_MYSQL_IDENT

 

OPENSHIFT_MYSQL_AIO

MYSQL_AIO

OPENSHIFT_MYSQL_MAX_ALLOWED_PACKET

MYSQL_MAX_ALLOWED_PACKET

OPENSHIFT_MYSQL_TABLE_OPEN_CACHE

MYSQL_TABLE_OPEN_CACHE

OPENSHIFT_MYSQL_SORT_BUFFER_SIZE

MYSQL_SORT_BUFFER_SIZE

OPENSHIFT_MYSQL_LOWER_CASE_TABLE_NAMES

MYSQL_LOWER_CASE_TABLE_NAMES

OPENSHIFT_MYSQL_MAX_CONNECTIONS

MYSQL_MAX_CONNECTIONS

OPENSHIFT_MYSQL_FT_MIN_WORD_LEN

MYSQL_FT_MIN_WORD_LEN

OPENSHIFT_MYSQL_FT_MAX_WORD_LEN

MYSQL_FT_MAX_WORD_LEN

OPENSHIFT_MYSQL_DEFAULT_STORAGE_ENGINE

 

OPENSHIFT_MYSQL_TIMEZONE

 
 

MYSQL_DATABASE

 

MYSQL_ROOT_PASSWORD

 

MYSQL_MASTER_USER

 

MYSQL_MASTER_PASSWORD

6.2.4. PostgreSQL

  1. Back up the v2 PostgreSQL database from the gear:

    $ rhc ssh -a <v2-application_name>
    $ mkdir ~/app-root/data/tmp
    $ pg_dump <database_name> | gzip > ~/app-root/data/tmp/<database_name>.gz
  2. Extract the backup file back to your local machine:

    $ rhc scp -a <v2_application_name> download <local_dest> app-root/data/tmp/<db-name>.gz
    $ gzip -d <database-name>.gz
    Note

    Save the backup file to a separate folder for step 4.

  3. Create the PostgreSQL service using the v2 application database name, user name and password to create the new service:

    $ oc new-app postgresql-persistent -p POSTGRESQL_DATABASE=dbname -p
    POSTGRESQL_PASSWORD=password -p POSTGRESQL_USER=username
  4. Check to see if the pod is ready to use:

    $ oc get pods
  5. When the pod is up and running, sync the backup directory to pod:

    $ oc rsync /local/path/to/dir <postgresql_pod_name>:/var/lib/pgsql/data
  6. Remotely access the pod:

    $ oc rsh <pod_name>
  7. Restore the database:

    psql dbname < /var/lib/pgsql/data/<database_backup_file>
  8. Remove all backup files that are no longer needed:

    $ rm /var/lib/pgsql/data/<database-backup-file>

Supported PostgreSQL Environment Variables

v2v3

OPENSHIFT_POSTGRESQL_DB_HOST

[service_name]_SERVICE_HOST

OPENSHIFT_POSTGRESQL_DB_PORT

[service_name]_SERVICE_PORT

OPENSHIFT_POSTGRESQL_DB_USERNAME

POSTGRESQL_USER

OPENSHIFT_POSTGRESQL_DB_PASSWORD

POSTGRESQL_PASSWORD

OPENSHIFT_POSTGRESQL_DB_LOG_DIR

 

OPENSHIFT_POSTGRESQL_DB_PID

 

OPENSHIFT_POSTGRESQL_DB_SOCKET_DIR

 

OPENSHIFT_POSTGRESQL_DB_URL

 

OPENSHIFT_POSTGRESQL_VERSION

 

OPENSHIFT_POSTGRESQL_SHARED_BUFFERS

 

OPENSHIFT_POSTGRESQL_MAX_CONNECTIONS

 

OPENSHIFT_POSTGRESQL_MAX_PREPARED_TRANSACTIONS

 

OPENSHIFT_POSTGRESQL_DATESTYLE

 

OPENSHIFT_POSTGRESQL_LOCALE

 

OPENSHIFT_POSTGRESQL_CONFIG

 

OPENSHIFT_POSTGRESQL_SSL_ENABLED

 
 

POSTGRESQL_DATABASE

 

POSTGRESQL_ADMIN_PASSWORD

6.2.5. MongoDB

Note
  • For OpenShift v3: MongoDB shell version 3.2.6
  • For OpenShift v2: MongoDB shell version 2.4.9
  1. Remotely access the v2 application via the ssh command:

    $ rhc ssh <v2_application_name>
  2. Run mongodump, specifying a single database with -d <database_name> -c <collections>. Without those options, dump all databases. Each database is dumped in its own directory:

    $ mongodump -h $OPENSHIFT_MONGODB_DB_HOST -o app-root/repo/mydbdump -u 'admin' -p $OPENSHIFT_MONGODB_DB_PASSWORD
    $ cd app-root/repo/mydbdump/<database_name>; tar -cvzf dbname.tar.gz
    $ exit
  3. Download dbdump to a local machine in the mongodump directory:

    $ mkdir mongodump
    $ rhc scp -a <v2 appname> download mongodump \
      app-root/repo/mydbdump/<dbname>/dbname.tar.gz
  4. Start a MongoDB pod in v3. Because the latest image (3.2.6) does not include mongo-tools, to use mongorestore or mongoimport commands you need to edit the default mongodb-persistent template to specify the image tag that contains the mongo-tools, “mongodb:2.4”. For that reason, the following oc export command and edit are necessary:

    $ oc export template mongodb-persistent -n openshift -o json > mongodb-24persistent.json

    Edit L80 of mongodb-24persistent.json; replace mongodb:latest with mongodb:2.4.

    $ oc new-app --template=mongodb-persistent -n <project-name-that-template-was-created-in> \
      MONGODB_USER=user_from_v2_app -p \
      MONGODB_PASSWORD=password_from_v2_db -p \
      MONGODB_DATABASE=v2_dbname -p \
      MONGODB_ADMIN_PASSWORD=password_from_v2_db
    $ oc get pods
  5. When the mongodb pod is up and running, copy the database archive files to the v3 MongoDB pod:

    $ oc rsync local/path/to/mongodump <mongodb_pod_name>:/var/lib/mongodb/data
    $ oc rsh <mongodb_pod>
  6. In the MongoDB pod, complete the following for each database you want to restore:

    $ cd /var/lib/mongodb/data/mongodump
    $ tar -xzvf dbname.tar.gz
    $ mongorestore -u $MONGODB_USER -p $MONGODB_PASSWORD -d dbname -v /var/lib/mongodb/data/mongodump
  7. Check if the database is restored:

    $ mongo admin -u $MONGODB_USER -p $MONGODB_ADMIN_PASSWORD
    $ use dbname
    $ show collections
    $ exit
  8. Remove the mongodump directory from the pod:

    $ rm -rf /var/lib/mongodb/data/mongodump

Supported MongoDB Environment Variables

v2v3

OPENSHIFT_MONGODB_DB_HOST

[service_name]_SERVICE_HOST

OPENSHIFT_MONGODB_DB_PORT

[service_name]_SERVICE_PORT

OPENSHIFT_MONGODB_DB_USERNAME

MONGODB_USER

OPENSHIFT_MONGODB_DB_PASSWORD

MONGODB_PASSWORD

OPENSHIFT_MONGODB_DB_URL

 

OPENSHIFT_MONGODB_DB_LOG_DIR

 
 

MONGODB_DATABASE

 

MONGODB_ADMIN_PASSWORD

 

MONGODB_NOPREALLOC

 

MONGODB_SMALLFILES

 

MONGODB_QUIET

 

MONGODB_REPLICA_NAME

 

MONGODB_KEYFILE_VALUE

6.3. Migrating Web Framework Applications

6.3.1. Overview

This topic reviews how to migrate Python, Ruby, PHP, Perl, Node.js, WordPress, Ghost, JBoss EAP, JBoss WS (Tomcat), and Wildfly 10 (JBoss AS) web framework applications from OpenShift version 2 (v2) to OpenShift version 3 (v3).

6.3.2. Python

  1. Set up a new GitHub repository and add it as a remote branch to the current, local v2 Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>.git
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. Ensure that all important files such as setup.py, wsgi.py, requirements.txt, and etc are pushed to new repository.

    • Ensure all required packages for your application are included in requirements.txt.
  4. Use the oc command to launch a new Python application from the builder image and source code:

    $ oc new-app --strategy=source
    python:3.3~https://github.com/<github-id>/<repo-name> --name=<app-name> -e
    <ENV_VAR_NAME>=<env_var_value>

Supported Python Versions

See Supported Container Images.

6.3.3. Ruby

  1. Set up a new GitHub repository and add it as a remote branch to the current, local v2 Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>.git
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. If you do not have a Gemfile and are running a simple rack application, copy this Gemfile into the root of your source:

    https://github.com/openshift/ruby-ex/blob/master/Gemfile
    Note

    The latest version of the rack gem that supports Ruby 2.0 is 1.6.4, so the Gemfile needs to be modified to gem 'rack', “1.6.4”.

    For Ruby 2.2 or later, use the rack gem 2.0 or later.

  4. Use the oc command to launch a new Ruby application from the builder image and source code:

    $ oc new-app --strategy=source
    ruby:2.0~https://github.com/<github-id>/<repo-name>.git

Supported Ruby Versions

See Supported Container Images.

6.3.4. PHP

  1. Set up a new GitHub repository and add it as a remote branch to the current, local v2 Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. Use the oc command to launch a new PHP application from the builder image and source code:

    $ oc new-app https://github.com/<github-id>/<repo-name>.git
    --name=<app-name> -e <ENV_VAR_NAME>=<env_var_value>

Supported PHP Versions

See Supported Container Images.

6.3.5. Perl

  1. Set up a new GitHub repository and add it as a remote branch to the current, local v2 Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. Edit the local Git repository and push changes upstream to make it v3 compatible:

    1. In v2, CPAN modules reside in .openshift/cpan.txt. In v3, the s2i builder looks for a file named cpanfile in the root directory of the source.

      $ cd <local-git-repository>
      $ mv .openshift/cpan.txt cpanfile

      Edit cpanfile, as it has a slightly different format:

      format of cpanfileformat of cpan.txt

      requires ‘cpan::mod’;

      cpan::mod

      requires ‘Dancer’;

      Dancer

      requires ‘YAML’;

      YAML

    2. Remove .openshift directory

      Note

      In v3, action_hooks and cron tasks are not supported in the same way. See Action Hooks for more information.

  4. Use the oc command to launch a new Perl application from the builder image and source code:
$ oc new-app https://github.com/<github-id>/<repo-name>.git

Supported Perl Versions

See Supported Container Images.

6.3.6. Node.js

  1. Set up a new GitHub repository and add it as a remote branch to the current, local Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. Edit the local Git repository and push changes upstream to make it v3 compatible:

    1. Remove the .openshift directory.

      Note

      In v3, action_hooks and cron tasks are not supported in the same way. See Action Hooks for more information.

    2. Edit server.js.

      • L116 server.js: 'self.app = express();'
      • L25 server.js: self.ipaddress = '0.0.0.0';
      • L26 server.js: self.port = 8080;

        Note

        Lines(L) are from the base V2 cartridge server.js.

  4. Use the oc command to launch a new Node.js application from the builder image and source code:

    $ oc new-app https://github.com/<github-id>/<repo-name>.git
    --name=<app-name> -e <ENV_VAR_NAME>=<env_var_value>

Supported Node.js Versions

See Supported Container Images.

Important

In OpenShift Online v3, version 0.10 is deprecated and no longer available to use.

6.3.7. WordPress

Important

Currently, support for migrating WordPress applications is offered by the community only and not by Red Hat support.

For guidance on migrating WordPress applications to OpenShift Online v3, see the OpenShift blog.

6.3.8. Ghost

Important

Currently, support for migrating Ghost applications is offered by the community only and not by Red Hat support.

For guidance on migrating Ghost applications to OpenShift Online v3, see the OpenShift blog.

6.3.9. JBoss EAP

Important

Currently, JBoss EAP is not available for OpenShift Online Starter. It is only available for OpenShift Online Pro.

  1. Set up a new GitHub repository and add it as a remote branch to the current, local Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. If the repository includes pre-built .war files, they need to reside in the deployments directory off the root directory of the repository.
  4. Create the new application using the JBoss EAP 7 builder image (jboss-eap70-openshift) and the source code repository from GitHub:

    $ oc new-app --strategy=source jboss-eap70-openshift:1.6~https://github.com/<github-id>/<repo-name>.git

6.3.10. JBoss WS (Tomcat)

  1. Set up a new GitHub repository and add it as a remote branch to the current, local Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. If the repository includes pre-built .war files, they need to reside in the deployments directory off the root directory of the repository.
  4. Create the new application using the JBoss Web Server 3 (Tomcat 7) builder image (jboss-webserver30-tomcat7) and the source code repository from GitHub:

    $ oc new-app --strategy=source
    jboss-webserver30-tomcat7-openshift~https://github.com/<github-id>/<repo-name>.git
    --name=<app-name> -e <ENV_VAR_NAME>=<env_var_value>

6.3.11. JBoss AS (Wildfly 10)

  1. Set up a new GitHub repository and add it as a remote branch to the current, local Git repository:

    $ git remote add <remote-name> https://github.com/<github-id>/<repo-name>
  2. Push the local v2 source code to the new repository:

    $ git push -u <remote-name> master
  3. Edit the local Git repository and push the changes upstream to make it v3 compatible:

    1. Remove .openshift directory.

      Note

      In v3, action_hooks and cron tasks are not supported in the same way. See Action Hooks for more information.

    2. Add the deployments directory to the root of the source repository. Move the .war files to ‘deployments’ directory.
  4. Use the the oc command to launch a new Wildfly application from the builder image and source code:

    $ oc new-app https://github.com/<github-id>/<repo-name>.git
     --image-stream=”openshift/wildfly:10.0" --name=<app-name> -e
     <ENV_VAR_NAME>=<env_var_value>
    Note

    The argument --name is optional to specify the name of your application. The argument -e is optional to add environment variables that are needed for build and deployment processes, such as OPENSHIFT_PYTHON_DIR.

6.3.12. Supported JBoss Versions

See Supported Container Images.

6.4. QuickStart Examples

6.4.1. Overview

Although there is no clear-cut migration path for v2 quickstart to v3 quickstart, the following quickstarts are currently available in v3. If you have an application with a database, rather than using oc new-app to create your application, then oc new-app again to start a separate database service and linking the two with common environment variables, you can use one of the following to instantiate the linked application and database at once, from your GitHub repository containing your source code. You can list all available templates with oc get templates -n openshift:

6.4.2. Workflow

Run a git clone of one of the above template URLs locally. Add and commit your application source code and push a GitHub repository, then start a v3 quickstart application from one of the templates listed above:

  1. Create a GitHub repository for your application.
  2. Clone a quickstart template and add your GitHub repository as a remote:

    $ git clone <one-of-the-template-URLs-listed-above>
    $ cd <your local git repository>
    $ git remote add upstream <https://github.com/<git-id>/<quickstart-repo>.git>
    $ git push -u upstream master
  3. Commit and push your source code to GitHub:

    $ cd <your local repository>
    $ git commit -am “added code for my app”
    $ git push origin master
  4. Create a new application in v3:

    $ oc new-app --template=<template> \
    -p SOURCE_REPOSITORY_URL=<https://github.com/<git-id>/<quickstart_repo>.git> \
    -p DATABASE_USER=<your_db_user> \
    -p DATABASE_NAME=<your_db_name> \
    -p DATABASE_PASSWORD=<your_db_password> \
    -p DATABASE_ADMIN_PASSWORD=<your_db_admin_password> 1
    1
    Only applicable for MongoDB.

    You should now have 2 pods running, a web framework pod, and a database pod. The web framework pod environment should match the database pod environment. You can list the environment variables with oc set env pod/<pod_name> --list:

    • DATABASE_NAME is now <DB_SERVICE>_DATABASE
    • DATABASE_USER is now <DB_SERVICE>_USER
    • DATABASE_PASSWORD is now <DB_SERVICE>_PASSWORD
    • DATABASE_ADMIN_PASSWORD is now MONGODB_ADMIN_PASSWORD (only applicable for MongoDB)

      If no SOURCE_REPOSITORY_URL is specified, the template will use the template URL (https://github.com/openshift/<quickstart>-ex) listed above as the source repository, and a hello-welcome application will be started.

  5. If you are migrating a database, export databases to a dump file and restore the database in the new v3 database pod. Refer to the steps outlined in Database Applications, skipping the oc new-app step as the database pod is already up and running.

6.5. Continuous Integration and Deployment (CI/CD)

6.5.1. Overview

This topic reviews the differences in continuous integration and deployment (CI/CD) applications between OpenShift version 2 (v2) and OpenShift version 3 (v3) and how to migrate these applications into the v3 environment.

6.5.2. Jenkins

The Jenkins applications in OpenShift version 2 (v2) and OpenShift version 3 (v3) are configured differently due to fundamental differences in architecture. For example, in v2, the application uses an integrated Git repository that is hosted in the gear to store the source code. In v3, the source code is located in a public or private Git repository that is hosted outside of the pod.

Furthermore, in OpenShift v3, Jenkins jobs can not only be triggered by source code changes, but also by changes in ImageStream, which are changes on the images that are used to build the application along with its source code. As a result, it is highly recommended that you migrate the Jenkins application manually by creating a new Jenkins application in v3, and then re-creating jobs with the configurations that are suitable to OpenShift v3 environment.

Consult these resources for more information on how to create a Jenkins application, configure jobs, and use Jenkins plug-ins properly:

6.6. Webhooks and Action Hooks

6.6.1. Overview

This topic reviews the differences in webhooks and action hooks between OpenShift version 2 (v2) and OpenShift version 3 (v3) and how to migrate these applications into the v3 environment.

6.6.2. Webhooks

  1. After creating a BuildConfig` from a GitHub repository, run:

    $ oc describe bc/<name-of-your-BuildConfig>

    This will output a webhook GitHub URL that looks like:

    <https://api.starter-us-east-1.openshift.com:443/oapi/v1/namespaces/nsname/buildconfigs/bcname/webhooks/secret/github>.
  2. Cut and paste this URL into GitHub, from the GitHub web console.
  3. In your GitHub repository, select Add Webhook from Settings → Webhooks & Services.
  4. Paste the URL output (similar to above) into the Payload URL field.

You should see a message from GitHub stating that your webhook was successfully configured.

Now, whenever you push a change to your GitHub repository, a new build will automatically start, and upon a successful build a new deployment will start.

Note

If you delete or recreate your application, you will have to update the Payload URL field in GitHub with the new BuildConfig webhook url.

6.6.3. Action Hooks

In OpenShift version 2 (v2), there are build, deploy, post_deploy, and pre_build scripts or action_hooks that are located in the .openshift/action_hooks directory. While there is no one-to-one mapping of function for these in v3, the S2I tool in v3 does have the option of adding customizable scripts, either in a designated URL or in the .s2i/bin directory of your source repository.

OpenShift version 3 (v3) also offers a post-build hook for running basic testing of an image after it is built and before it is pushed to the registry. Deployment hooks are configured in the deployment configuration.

In v2, action_hooks are commonly used to set up environment variables. In v2, any environment variables should be passed with:

$ oc new-app <source-url> -e ENV_VAR=env_var

or:

$ oc new-app <template-name> -p ENV_VAR=env_var

Also, environment variables can be added or changed using:

$ oc set env dc/<name-of-dc>
ENV_VAR1=env_var1 ENV_VAR2=env_var2’

6.7. S2I Tool

6.7.1. Overview

The Source-to-Image (S2I) tool injects application source code into a container image and the final product is a new and ready-to-run container image that incorporates the builder image and built source code. The S2I tool can be installed on your local machine without OpenShift Online from the repository.

The S2I tool is a very powerful tool to test and verify your application and images locally before using them on OpenShift Online.

6.7.2. Creating a Container Image

  1. Identify the builder image that is needed for the application. Red Hat offers multiple builder images for different languages including Python, Ruby, Perl, PHP, and Node.js. Other images are available from the community space.
  2. S2I can build images from source code in a local file system or from a Git repository. To build a new container image from the builder image and the source code:

    $ s2i build <source-location> <builder-image-name> <output-image-name>
    Note

    <source-location> can either be a Git repository URL or a directory to source code in a local file system.

  3. Test the built image with the Docker daemon:

    $ docker run -d --name <new-name> -p <port-number>:<port-number> <output-image-name>
    $ curl localhost:<port-number>

6.8. Support Guide

6.8.1. Overview

This topic reviews supported languages, frameworks, databases, and markers for OpenShift version 2 (v2) and OpenShift version 3 (v3).

6.8.2. Supported Databases

See the Supported Databases section of the Database Applications topic.

6.8.3. Supported Languages

6.8.4. Supported Frameworks

Table 6.1. Supported Frameworks

v2v3

Jenkins Server

jenkins-persistent

Drupal 7

 

Ghost 0.7.5

 

WordPress 4

 

Ceylon

 

Go

 

MEAN

 

6.8.5. Supported Markers

Table 6.2. Python

v2v3

pip_install

If your repository contains requirements.txt, then pip is invoked by default. Otherwise, pip is not used.

Table 6.3. Ruby

v2v3

disable_asset_compilation

This can be done by setting DISABLE_ASSET_COMPILATION environment variable to true on the buildconfig strategy definition.

Table 6.4. Perl

v2v3

enable_cpan_tests

This can be done by setting ENABLE_CPAN_TEST environment variable to true on the build configuration.

Table 6.5. PHP

v2v3

use_composer

composer is always used if the source repository includes a composer.json in the root directory.

Table 6.6. Node.js

v2v3

NODEJS_VERSION

N/A

use_npm

npm is always used to start the application, unless DEV_MODE is set to true, in which case nodemon is used instead.

Table 6.7. JBoss EAP, JBoss WS, WildFly

v2v3

enable_debugging

This option is controlled via the ENABLE_JPDA environment variable set on the deployment configuration by setting it to any non-empty value.

skip_maven_build

If pom.xml is present, maven will be run.

java7

N/A

java8

JavaEE is using JDK8.

Table 6.8. Jenkins

v2v3

enable_debugging

N/A

Table 6.9. All

v2v3

force_clean_build

There is a similar concept in v3, as noCache field in buildconfig forces the container build to rerun each layer. In the S2I build, the incremental flag is false by default, which indicates a clean build.

hot_deploy

Ruby, Python, Perl, PHP, Node.js

enable_public_server_status

N/A

disable_auto_scaling

Autoscaling is off by default and it can be turn on via pod auto-scaling.

6.8.6. Supported Environment Variables

Chapter 7. Tutorials

7.1. Overview

This topic group includes information on how to get your application up and running in OpenShift Online and covers different languages and their frameworks.

7.2. Quickstart Templates

7.2.1. Overview

A quickstart is a basic example of an application running on OpenShift Online. Quickstarts come in a variety of languages and frameworks, and are defined in a template, which is constructed from a set of services, build configurations, and deployment configurations. This template references the necessary images and source repositories to build and deploy the application.

To explore a quickstart, create an application from a template. Your administrator may have already installed these templates in your OpenShift Online cluster, in which case you can simply select it from the web console. See the template documentation for more information on how to upload, create from, and modify a template.

Quickstarts refer to a source repository that contains the application source code. To customize the quickstart, fork the repository and, when creating an application from the template, substitute the default source repository name with your forked repository. This results in builds that are performed using your source code instead of the provided example source. You can then update the code in your source repository and launch a new build to see the changes reflected in the deployed application.

7.2.2. Web Framework Quickstart Templates

These quickstarts provide a basic application of the indicated framework and language:

7.3. Ruby on Rails

7.3.1. Overview

Ruby on Rails is a popular web framework written in Ruby. This guide covers using Rails 4 on OpenShift Online.

Warning

We strongly advise going through the whole tutorial to have an overview of all the steps necessary to run your application on the OpenShift Online. If you experience a problem try reading through the entire tutorial and then going back to your issue. It can also be useful to review your previous steps to ensure that all the steps were executed correctly.

For this guide you will need:

  • Basic Ruby/Rails knowledge
  • Locally installed version of Ruby 2.0.0+, Rubygems, Bundler
  • Basic Git knowledge
  • Provisioned account in OpenShift Online

7.3.2. Local Workstation Setup

First make sure that an instance of OpenShift Online is running and is available. Also make sure that your oc CLI client is installed and the command is accessible from your command shell, so you can use it to log in using your email address and password.

7.3.2.1. Setting Up the Database

Rails applications are almost always used with a database. For the local development we chose the PostgreSQL database. To install it type:

$ sudo yum install -y postgresql postgresql-server postgresql-devel

Next you need to initialize the database with:

$ sudo postgresql-setup initdb

This command will create the /var/lib/pgsql/data directory, in which the data will be stored.

Start the database by typing:

$ sudo systemctl start postgresql.service

When the database is running, create your rails user:

$ sudo -u postgres createuser -s rails

Note that the user we created has no password.

7.3.3. Writing Your Application

If you are starting your Rails application from scratch, you need to install the Rails gem first.

$ gem install rails
Successfully installed rails-4.2.0
1 gem installed

After you install the Rails gem create a new application, with PostgreSQL as your database:

$ rails new rails-app --database=postgresql

Then change into your new application directory.

$ cd rails-app

If you already have an application, make sure the pg (postgresql) gem is present in your Gemfile. If not edit your Gemfile by adding the gem:

gem 'pg'

To generate a new Gemfile.lock with all your dependencies run:

$ bundle install

In addition to using the postgresql database with the pg gem, you’ll also need to ensure the config/database.yml is using the postgresql adapter.

Make sure you updated default section in the config/database.yml file, so it looks like this:

default: &default
  adapter: postgresql
  encoding: unicode
  pool: 5
  host: localhost
  username: rails
  password:

Create your application’s development and test databases by using this rake command:

$ rake db:create

This will create development and test database in your PostgreSQL server.

7.3.3.1. Creating a Welcome Page

Since Rails 4 no longer serves a static public/index.html page in production, we need to create a new root page.

In order to have a custom welcome page we need to do following steps:

  • Create a controller with an index action
  • Create a view page for the welcome controller index action
  • Create a route that will serve applications root page with the created controller and view

Rails offers a generator that will do all this necessary steps for you.

$ rails generate controller welcome index

All the necessary files have been created, now we just need to edit line 2 in config/routes.rb file to look like:

root 'welcome#index'

Run the rails server to verify the page is available.

$ rails server

You should see your page by visiting http://localhost:3000 in your browser. If you don’t see the page, check the logs that are output to your server to debug.

7.3.3.2. Configuring the Application for OpenShift Online

In order to have your application communicating with the PostgreSQL database service that will be running in OpenShift Online, you will need to edit the default section in your config/database.yml to use environment variables, which you will define later, upon the database service creation.

The default section in your edited config/database.yml together with pre-defined variables should look like:

<% user = ENV.key?("POSTGRESQL_ADMIN_PASSWORD") ? "root" : ENV["POSTGRESQL_USER"] %>
<% password = ENV.key?("POSTGRESQL_ADMIN_PASSWORD") ? ENV["POSTGRESQL_ADMIN_PASSWORD"] : ENV["POSTGRESQL_PASSWORD"] %>
<% db_service = ENV.fetch("DATABASE_SERVICE_NAME","").upcase %>

default: &default
  adapter: postgresql
  encoding: unicode
  # For details on connection pooling, see rails configuration guide
  # http://guides.rubyonrails.org/configuring.html#database-pooling
  pool: <%= ENV["POSTGRESQL_MAX_CONNECTIONS"] || 5 %>
  username: <%= user %>
  password: <%= password %>
  host: <%= ENV["#{db_service}_SERVICE_HOST"] %>
  port: <%= ENV["#{db_service}_SERVICE_PORT"] %>
  database: <%= ENV["POSTGRESQL_DATABASE"] %>

For an example of how the final file should look, see Ruby on Rails example application config/database.yml.

7.3.3.3. Storing Your Application in Git

OpenShift Online requires git, if you don’t have it installed you will need to install it.

Building an application in OpenShift Online usually requires that the source code be stored in a git repository, so you will need to install git if you do not already have it.

Make sure you are in your Rails application directory by running the ls -1 command. The output of the command should look like:

$ ls -1
app
bin
config
config.ru
db
Gemfile
Gemfile.lock
lib
log
public
Rakefile
README.rdoc
test
tmp
vendor

Now run these commands in your Rails app directory to initialize and commit your code to git:

$ git init
$ git add .
$ git commit -m "initial commit"

Once your application is committed you need to push it to a remote repository. For this you would need a GitHub account, in which you create a new repository.

Set the remote that points to your git repository:

$ git remote add origin git@github.com:<namespace/repository-name>.git

After that, push your application to your remote git repository.

$ git push

7.3.4. Deploying Your Application to OpenShift Online

After creating the the rails-app project, you will be automatically switched to the new project namespace.

Deploying your application in OpenShift Online involves three steps:

  • Creating a database service from OpenShift Online’s PostgreSQL image
  • Creating a frontend service from OpenShift Online’s Ruby 2.0 builder image and your Ruby on Rails source code, which we wire with the database service
  • Creating a route for your application.

7.3.4.1. Creating the Database Service

Your Rails application expects a running database service. For this service use PostgeSQL database image.

To create the database service you will use the oc new-app command. To this command you will need to pass some necessary environment variables which will be used inside the database container. These environment variables are required to set the username, password, and name of the database. You can change the values of these environment variables to anything you would like. The variables we are going to be setting are as follows:

  • POSTGRESQL_DATABASE
  • POSTGRESQL_USER
  • POSTGRESQL_PASSWORD

Setting these variables ensures:

  • A database exists with the specified name
  • A user exists with the specified name
  • The user can access the specified database with the specified password

For example:

$ oc new-app postgresql -e POSTGRESQL_DATABASE=db_name -e POSTGRESQL_USER=username -e POSTGRESQL_PASSWORD=password

To also set the password for the database administrator, append to the previous command with:

-e POSTGRESQL_ADMIN_PASSWORD=admin_pw

To watch the progress of this command:

$ oc get pods --watch

7.3.4.2. Creating the Frontend Service

To bring your application to OpenShift Online, you need to specify a repository in which your application lives, using once again the oc new-app command, in which you will need to specify database related environment variables we setup in the Creating the Database Service:

$ oc new-app path/to/source/code --name=rails-app -e POSTGRESQL_USER=username -e POSTGRESQL_PASSWORD=password -e POSTGRESQL_DATABASE=db_name

With this command, OpenShift Online fetches the source code, sets up the builder image, builds your application image, and deploys the newly created image together with the specified environment variables. The application is named rails-app.

You can verify the environment variables have been added by viewing the JSON document of the rails-app DeploymentConfig:

$ oc get dc rails-app -o json

You should see the following section:

env": [
    {
        "name": "POSTGRESQL_USER",
        "value": "username"
    },
    {
        "name": "POSTGRESQL_PASSWORD",
        "value": "password"
    },
    {
        "name": "POSTGRESQL_DATABASE",
        "value": "db_name"
    }
],

To check the build process:

$ oc logs -f build rails-app-1

Once the build is complete, you can look at the running pods in OpenShift Online.

$ oc get pods

You should see a line starting with myapp-<number>-<hash>, and that is your application running in OpenShift Online.

Before your application will be functional, you need to initialize the database by running the database migration script. There are two ways you can do this:

  • Manually from the running frontend container:

First you need to exec into frontend container with rsh command:

$ oc rsh <FRONTEND_POD_ID>

Run the migration from inside the container:

$ RAILS_ENV=production bundle exec rake db:migrate

If you are running your Rails application in a development or test environment you don’t have to specify the RAILS_ENV environment variable.

7.3.4.3. Creating a Route for Your Application

Expose the frontend service by typing:

$ oc expose service rails-app

7.4. Setting Up a Nexus Mirror for Maven

7.4.1. Introduction

While developing your application with Java and Maven, you will most likely be building many times. In order to shorten the build times of your pods, Maven dependencies can be cached in a local Nexus repository. This tutorial will guide you through creating a Nexus repository on your cluster.

This tutorial assumes that you are working with a project that is already set up for use with Maven. If you are interested in using Maven with your Java project, it is highly recommended that you look at their guide.

In addition, be sure to check your application’s image for Maven mirror capabilities. Many images that use Maven have a MAVEN_MIRROR_URL environment variable that you can use to simplify this process. If it does not have this capability, read the Nexus documentation to configure your build properly.

Furthermore, make sure that you give each pod enough resources to function. You may have to edit the pod template in the Nexus deployment configuration to request more resources.

7.4.2. Setting up Nexus

  1. Download and deploy the official Nexus container image:

    oc new-app sonatype/nexus
  2. Create a route by exposing the newly created Nexus service:

    oc expose svc/nexus
  3. Use oc get routes to find the pod’s new external address.

    oc get routes

    The output should resemble:

    NAME      HOST/PORT                              PATH      SERVICES   PORT       TERMINATION
    nexus     nexus-myproject.192.168.1.173.xip.io             nexus      8081-tcp
  4. Confirm that Nexus is running by navigating your browser to the URL under HOST/PORT. To sign in to Nexus, the default administrator username is admin, and the password is admin123.
Note

Nexus comes pre-configured for the Central Repository, but you may need others for your application. For many Red Hat images, it is recommended to add the jboss-ga repository at Maven repository.

7.4.2.1. Using Probes to Check for Success

This is a good time to set up readiness and liveness probes. These will periodically check to see that Nexus is running properly.

$ oc set probe dc/nexus \
	--liveness \
	--failure-threshold 3 \
	--initial-delay-seconds 30 \
	-- echo ok
$ oc set probe dc/nexus \
	--readiness \
	--failure-threshold 3 \
	--initial-delay-seconds 30 \
	--get-url=http://:8081/nexus/content/groups/public

7.4.2.2. Adding Persistence to Nexus

Note

If you do not want persistent storage, continue to Connecting to Nexus. However, your cached dependencies and any configuration customization will be lost if the pod is restarted for any reason.

Create a persistent volume claim (PVC) for Nexus, so that the cached dependencies are not lost when the pod running the server terminates. PVCs require available persistent volumes (PV) in the cluster. If there are no PVs available and you do not have administrator access on your cluster, ask your system administrator to create a Read/Write Persistent Volume for you.

Add a PVC to the Nexus deployment configuration.

$ oc volumes dc/nexus --add \
	--name 'nexus-volume-1' \
	--type 'pvc' \
	--mount-path '/sonatype-work/' \
	--claim-name 'nexus-pv' \
	--claim-size '1G' \
	--overwrite

This removes the previous emptyDir volume for the deployment config and adds a claim for one gigabyte of persistent storage mounted at /sonatype-work, which is where the dependencies will be stored. Due to the change in configuration, the Nexus pod will be redeployed automatically.

To verify that Nexus is running, refresh the Nexus page in your browser. You can monitor the deployment’s progress using:

$ oc get pods -w

7.4.3. Connecting to Nexus

The next steps demonstrate defining a build that uses the new Nexus repository. The rest of the tutorial uses this example repository with wildfly-100-centos7 as a builder, but these changes should work for any project.

The example builder image supports MAVEN_MIRROR_URL as part of its environment, so we can use this to point our builder image to our Nexus repository. If your image does not support consuming an environment variable to configure a Maven mirror, you may need to modify the builder image to provide the correct Maven settings to point to the Nexus mirror.

$ oc new-build openshift/wildfly-100-centos7:latest~https://github.com/openshift/jee-ex.git \
	-e MAVEN_MIRROR_URL='http://nexus.<Nexus_Project>:8081/nexus/content/groups/public'
$ oc logs build/jee-ex-1 --follow

Replace <Nexus_Project> with the project name of the Nexus repository. If it is in the same project as the application that is using it, you can remove the <Nexus_Project>.. Learn more about DNS resolution in OpenShift Online.

7.4.4. Confirming Success

In your web browser, navigate to http://<Nexus IP>:8081/nexus/content/groups/public to confirm that it has stored your application’s dependencies. You can also check the build logs to see if Maven is using the Nexus mirror. If successful, you should see output referencing the URL http://nexus:8081.

7.4.5. Additional Resources

7.5. Binary Builds

7.5.1. Introduction

The binary build feature in OpenShift allows developers to upload source or artifacts directly to a build instead of having the build pull source from a Git repository URL. Any BuildConfig with a strategy of source, Docker, or custom may be started as a binary build. When starting a build from local artifacts, the existing source reference is replaced with the source coming from the local user’s machine.

The source may be supplied in several ways which correspond to arguments available when using the start-build command:

  • From a file (--from-file): This is the case when the entire source of the build consists of a single file. For example, it may be a Dockerfile for a Docker build, pom.xml for a Wildfly build, or Gemfile for a Ruby build.
  • From a directory (--from-directory): Use this when the source is in a local directory and is not committed to a Git repository. The start-build command will create an archive of the given directory and upload it to the builder as source.
  • From an archive (--from-archive): Use this when an archive with the source already exists. The archive may be in either tar, tar.gz, or zip format.
  • From a Git repository (--from-repo): This is for source that is currently part of a Git repository on the user’s local machine. The HEAD commit of the current repository will be archived and sent to OpenShift for building.

7.5.1.1. Use Cases

Binary builds remove the requirement for a build to pull source from an existing Git repository. Reasons to use binary builds include:

  • Building and testing local code changes. Source from a public repository can be cloned and local changes can be uploaded to OpenShift for building. Local changes do not have to be committed or pushed anywhere.
  • Building private code. New builds can be started from scratch as binary builds. The source can then be uploaded directly from your local workstation to OpenShift without having to check it in to an SCM.
  • Building images with artifacts from other sources. With Jenkins pipelines, binary builds are useful to combine artifacts built with tools such as Maven or C compiler, and runtime images that make use of those builds.

7.5.1.2. Limitations

  • Binary builds are not repeatable. Because binary builds rely on the user uploading artifacts at build start, OpenShift cannot repeat the same build without the user repeating the same upload every time.
  • Binary builds cannot be triggered automatically. They can only be started manually when the user uploads the required binary artifacts.
Note

Builds that are started as binary builds may also have a configured source URL. If that’s the case, triggers will successfully launch the build but source will come from the configured source URL and not from what was supplied by the user the last time the build ran.

7.5.2. Tutorials Overview

The following tutorials assume that you have an OpenShift cluster available and that you have a project where you can create artifacts. It requires that you have both git and oc available locally.

7.5.2.1. Tutorial: Building local code changes

  1. Create a new application based on an existing source repository and create a route for it:

    $ oc new-app https://github.com/openshift/ruby-hello-world.git
    $ oc expose svc/ruby-hello-world
  2. Wait for the initial build to complete and view the application’s page by navigating to the route’s host. You should get a welcome page:

    $ oc get route ruby-hello-world
  3. Clone the repository locally:

    $ git clone https://github.com/openshift/ruby-hello-world.git
    $ cd ruby-hello-world
  4. Make a change to the application’s view. Using your favorite editor, edit views/main.rb: Change the <body> tag to <body style="background-color:blue">.
  5. Start a new build with your locally-modified source. From the repository’s local directory, run:

    ----
    $ oc start-build ruby-hello-world --from-dir="." --follow
    ----

Once your build has completed and the application has redeployed, navigating to the application’s route host should result in a page with a blue background.

You can keep making changes locally and building your code with oc start-build --from-dir.

You can also create a branch of the code, commit your changes locally, and use the repository’s HEAD as the source for your build:

$ git checkout -b my_branch
$ git add .
$ git commit -m "My changes"
$ oc start-build ruby-hello-world --from-repo="." --follow

7.5.2.2. Tutorial: Building private code

  1. Create a local directory to hold your code:

    $ mkdir myapp
    $ cd myapp
  2. In the directory create a file named Dockerfile with the following content:

    FROM centos:centos7
    
    EXPOSE 8080
    
    COPY index.html /var/run/web/index.html
    
    CMD cd /var/run/web && python -m SimpleHTTPServer 8080
  3. Create a file named index.html with the following content:

    <html>
      <head>
        <title>My local app</title>
      </head>
      <body>
        <h1>Hello World</h1>
        <p>This is my local application</p>
      </body>
    </html>
  4. Create a new build for your application:

    $ oc new-build --strategy docker --binary --docker-image centos:centos7 --name myapp
  5. Start a binary build using the local directory’s content:

    $ oc start-build myapp --from-dir . --follow
  6. Deploy the application using new-app, then create a route for it:

    $ oc new-app myapp
    $ oc expose svc/myapp
  7. Get the host name for your route and navigate to it:

    $ oc get route myapp

After having built and deployed your code, you can iterate by making changes to your local files and starting new builds by invoking oc start-build myapp --from-dir. Once built, the code will be automatically deployed and the changes will be reflected in your browser when you refresh the page.

7.5.2.3. Tutorial: Binary artifacts from pipeline

Jenkins on OpenShift allows using slave images with the appropriate tools to build your code. For example, you can use the maven slave to build a WAR from your code repository. However, once this artifact is built, you need to commit it to an image that contains the right runtime artifacts to run your code. A binary build may be used to add these artifacts to your runtime image. In the following tutorial, we’ll create a Jenkins pipeline that makes use of the maven slave to build a WAR, and then uses a binary build with a Dockerfile to add that WAR to a wildfly runtime image.

  1. Create a new directory for your application:

    $ mkdir mavenapp
    $ cd mavenapp
  2. Create a Dockerfile that copies a WAR to the appropriate location inside a wildfly image for execution. Copy the following to a local file named Dockerfile:

    FROM wildfly:latest
    COPY ROOT.war /wildfly/standalone/deployments/ROOT.war
    CMD  $STI_SCRIPTS_PATH/run
  3. Create a new BuildConfig for that Dockerfile:

    Note

    This will automatically start a build that will initially fail because the ROOT.war artifact is not yet available. The pipeline below will pass that WAR to the build using a binary build.

    $ cat Dockerfile | oc new-build -D - --name mavenapp
  4. Create a BuildConfig with the Jenkins pipeline that will build a WAR and then use that WAR to build an image using the previously created Dockerfile. The same pattern can be used for other platforms where a binary artifact is built by a set of tools and is then combined with a different runtime image for the final package. Save the following code to mavenapp-pipeline.yml:

    apiVersion: v1
    kind: BuildConfig
    metadata:
      name: mavenapp-pipeline
    spec:
      strategy:
        jenkinsPipelineStrategy:
          jenkinsfile: |-
            pipeline {
              agent { label "maven" }
              stages {
                stage("Clone Source") {
                  steps {
                    checkout([$class: 'GitSCM',
                                branches: [[name: '*/master']],
                                extensions: [
                                  [$class: 'RelativeTargetDirectory', relativeTargetDir: 'mavenapp']
                                ],
                                userRemoteConfigs: [[url: 'https://github.com/openshift/openshift-jee-sample.git']]
                            ])
                  }
                }
                stage("Build WAR") {
                  steps {
                    dir('mavenapp') {
                      sh 'mvn clean package -Popenshift'
                    }
                  }
                }
                stage("Build Image") {
                  steps {
                    dir('mavenapp/target') {
                      sh 'oc start-build mavenapp --from-dir . --follow'
                    }
                  }
                }
              }
            }
        type: JenkinsPipeline
      triggers: []
  5. Create the pipeline build. If Jenkins is not deployed to your project, creating the BuildConfig with the pipeline will result in Jenkins getting deployed. It may take a couple of minutes before Jenkins is ready to build your pipeline. You can check the status of the Jenkins rollout by invoking, oc rollout status dc/jenkins:

    $ oc create -f ./mavenapp-pipeline.yml
  6. Once Jenkins is ready, start the pipeline defined previously:

    $ oc start-build mavenapp-pipeline
  7. When the pipeline has finished building, deploy the new application using new-app and expose its route:

    $ oc new-app mavenapp
    $ oc expose svc/mavenapp
  8. Using your browser, navigate to the route for the application:

    $ oc get route mavenapp

Chapter 8. Builds

8.1. How Builds Work

8.1.1. What Is a Build?

A build in OpenShift Online is the process of transforming input parameters into a resulting object. Most often, builds are used to transform source code into a runnable container image.

A build configuration, or BuildConfig, is characterized by a build strategy and one or more sources. The strategy determines the aforementioned process, while the sources provide its input.

The supported build strategies are:

And there are six types of sources that can be given as build input:

It is up to each build strategy to consider or ignore a certain type of source, as well as to determine how it is to be used. Binary and Git are mutually exclusive source types. Image can be used by itself or together with either Git or Binary. The Binary source type is unique from the other options in how it is specified to the system.

8.1.2. What Is a BuildConfig?

A build configuration describes a single build definition and a set of triggers for when a new build should be created. Build configurations are defined by a BuildConfig, which is a REST object that can be used in a POST to the API server to create a new instance.

Depending on how you choose to create your application using OpenShift Online, a BuildConfig is typically generated automatically for you if you use the web console or CLI, and it can be edited at any time. Understanding the parts that make up a BuildConfig and their available options can help if you choose to manually tweak your configuration later.

The following example BuildConfig results in a new build every time a container image tag or the source code changes:

BuildConfig Object Definition

kind: "BuildConfig"
apiVersion: "v1"
metadata:
  name: "ruby-sample-build" 1
spec:
  runPolicy: "Serial" 2
  triggers: 3
    -
      type: "GitHub"
      github:
        secret: "secret101"
    - type: "Generic"
      generic:
        secret: "secret101"
    -
      type: "ImageChange"
  source: 4
    git:
      uri: "https://github.com/openshift/ruby-hello-world"
  strategy: 5
    sourceStrategy:
      from:
        kind: "ImageStreamTag"
        name: "ruby-20-centos7:latest"
  output: 6
    to:
      kind: "ImageStreamTag"
      name: "origin-ruby-sample:latest"
  postCommit: 7
      script: "bundle exec rake test"

1
This specification will create a new BuildConfig named ruby-sample-build.
2
The runPolicy field controls whether builds created from this build configuration can be run simultaneously. The default value is Serial, which means new builds will run sequentially, not simultaneously.
3
You can specify a list of triggers, which cause a new build to be created.
4
The source section defines the source of the build. The source type determines the primary source of input, and can be either Git, to point to a code repository location, or Binary, to accept binary payloads. It is possible to have multiple sources at once, refer to the documentation for each source type for details.
5
The strategy section describes the build strategy used to execute the build. You can specify a Source strategy here. This above example uses the ruby-20-centos7 container image that Source-To-Image will use for the application build.
6
After the container image is successfully built, it will be pushed into the repository described in the output section.
7
The postCommit section defines an optional build hook.

8.2. Basic Build Operations

8.2.1. Starting a Build

Manually start a new build from an existing build configuration in your current project using the following command:

$ oc start-build <buildconfig_name>

Re-run a build using the --from-build flag:

$ oc start-build --from-build=<build_name>

Specify the --follow flag to stream the build’s logs in stdout:

$ oc start-build <buildconfig_name> --follow

Specify the --env flag to set any desired environment variable for the build:

$ oc start-build <buildconfig_name> --env=<key>=<value>

Rather than relying on a Git source pull for a build, you can can also start a build by directly pushing your source, which could be the contents of a Git or SVN working directory, a set of prebuilt binary artifacts you want to deploy, or a single file. This can be done by specifying one of the following options for the start-build command:

OptionDescription

--from-dir=<directory>

Specifies a directory that will be archived and used as a binary input for the build.

--from-file=<file>

Specifies a single file that will be the only file in the build source. The file is placed in the root of an empty directory with the same file name as the original file provided.

--from-repo=<local_source_repo>

Specifies a path to a local repository to use as the binary input for a build. Add the --commit option to control which branch, tag, or commit is used for the build.

When passing any of these options directly to the build, the contents are streamed to the build and override the current build source settings.

Note

Builds triggered from binary input will not preserve the source on the server, so rebuilds triggered by base image changes will use the source specified in the build configuration.

For example, the following command sends the contents of a local Git repository as an archive from the tag v2 and starts a build:

$ oc start-build hello-world --from-repo=../hello-world --commit=v2

8.2.2. Canceling a Build

Manually cancel a build using the web console, or with the following CLI command:

$ oc cancel-build <build_name>

Cancel multiple builds at the same time:

$ oc cancel-build <build1_name> <build2_name> <build3_name>

Cancel all builds created from the build configuration:

$ oc cancel-build bc/<buildconfig_name>

Cancel all builds in a given state (for example, new or pending), ignoring the builds in other states:

$ oc cancel-build bc/<buildconfig_name>  --state=<state>

8.2.3. Deleting a BuildConfig

Delete a BuildConfig using the following command:

$ oc delete bc <BuildConfigName>

This will also delete all builds that were instantiated from this BuildConfig. Specify the --cascade=false flag if you do not want to delete the builds:

$ oc delete --cascade=false bc <BuildConfigName>

8.2.4. Viewing Build Details

You can view build details with the web console or by using the oc describe CLI command:

$ oc describe build <build_name>

This displays information such as:

  • The build source
  • The build strategy
  • The output destination
  • Digest of the image in the destination registry
  • How the build was created

If the build uses the Source strategy, the oc describe output also includes information about the source revision used for the build, including the commit ID, author, committer, and message.

8.2.5. Accessing Build Logs

You can access build logs using the web console or the CLI.

To stream the logs using the build directly:

$ oc logs -f build/<build_name>

To stream the logs of the latest build for a build configuration:

$ oc logs -f bc/<buildconfig_name>

To return the logs of a given version build for a build configuration:

$ oc logs --version=<number> bc/<buildconfig_name>

Log Verbosity

To enable more verbose output, pass the BUILD_LOGLEVEL environment variable as part of the sourceStrategy in a BuildConfig:

sourceStrategy:
...
  env:
    - name: "BUILD_LOGLEVEL"
      value: "2" 1
1
Adjust this value to the desired log level.
Note

A platform administrator can set the default build verbosity for the entire OpenShift Online instance by configuring env/BUILD_LOGLEVEL for the BuildDefaults admission controller. This default can be overridden by specifying BUILD_LOGLEVEL in a given BuildConfig. You can specify a higher priority override on the command line for non-binary builds by passing --build-loglevel to oc start-build.

Available log levels for Source builds are as follows:

Level 0

Produces output from containers running the assemble script and all encountered errors. This is the default.

Level 1

Produces basic information about the executed process.

Level 2

Produces very detailed information about the executed process.

Level 3

Produces very detailed information about the executed process, and a listing of the archive contents.

Level 4

Currently produces the same information as level 3.

Level 5

Produces everything mentioned on previous levels and additionally provides docker push messages.

8.3. Build Inputs

8.3.1. How Build Inputs Work

A build input provides source content for builds to operate on. There are several ways to provide source in OpenShift Online. In order of precedence:

Important

The Docker build strategy is not supported in OpenShift Online. Therefore, inline Dockerfile definitions are not accepted.

Different inputs can be combined into a single build. Binary input and Git repositories are mutually exclusive inputs.

Input secrets are useful for when you do not want certain resources or credentials used during a build to be available in the final application image produced by the build, or want to consume a value that is defined in a Secret resource. External artifacts can be used to pull in additional files that are not available as one of the other build input types.

Whenever a build is run:

  1. A working directory is constructed and all input content is placed in the working directory. For example, the input Git repository is cloned into the working directory, and files specified from input images are copied into the working directory using the target path.
  2. The build process changes directories into the contextDir, if one is defined.
  3. The content from the current directory is provided to the build process for reference by the assemble script. This means any input content that resides outside the contextDir will be ignored by the build.

The following example of a source definition includes multiple input types and an explanation of how they are combined. For more details on how each input type is defined, see the specific sections for each input type.

source:
  git:
    uri: https://github.com/openshift/ruby-hello-world.git 1
  images:
  - from:
      kind: ImageStreamTag
      name: myinputimage:latest
      namespace: mynamespace
    paths:
    - destinationDir: app/dir/injected/dir 2
      sourcePath: /usr/lib/somefile.jar
  contextDir: "app/dir" 3
1
The repository to be cloned into the working directory for the build.
2
/usr/lib/somefile.jar from myinputimage will be stored in <workingdir>/app/dir/injected/dir.
3
The working directory for the build will become <original_workingdir>/app/dir.

8.3.2. Image Source

Additional files can be provided to the build process via images. Input images are referenced in the same way the From and To image targets are defined. This means both container images and image stream tags can be referenced. In conjunction with the image, you must provide one or more path pairs to indicate the path of the files or directories to copy the image and the destination to place them in the build context.

The source path can be any absolute path within the image specified. The destination must be a relative directory path. At build time, the image will be loaded and the indicated files and directories will be copied into the context directory of the build process. This is the same directory into which the source repository content (if any) is cloned. If the source path ends in /. then the content of the directory will be copied, but the directory itself will not be created at the destination.

Image inputs are specified in the source definition of the BuildConfig:

source:
  git:
    uri: https://github.com/openshift/ruby-hello-world.git
  images: 1
  - from: 2
      kind: ImageStreamTag
      name: myinputimage:latest
      namespace: mynamespace
    paths: 3
    - destinationDir: injected/dir 4
      sourcePath: /usr/lib/somefile.jar 5
  - from:
      kind: ImageStreamTag
      name: myotherinputimage:latest
      namespace: myothernamespace
    pullSecret: mysecret 6
    paths:
    - destinationDir: injected/dir
      sourcePath: /usr/lib/somefile.jar
1
An array of one or more input images and files.
2
A reference to the image containing the files to be copied.
3
An array of source/destination paths.
4
The directory relative to the build root where the build process can access the file.
5
The location of the file to be copied out of the referenced image.
6
An optional secret provided if credentials are needed to access the input image.

8.3.3. Git Source

When specified, source code will be fetched from the location supplied.

The source definition is part of the spec section in the BuildConfig:

source:
  git: 1
    uri: "https://github.com/openshift/ruby-hello-world"
    ref: "master"
  contextDir: "app/dir" 2
1
The git field contains the URI to the remote Git repository of the source code. Optionally, specify the ref field to check out a specific Git reference. A valid ref can be a SHA1 tag or a branch name.
2
The contextDir field allows you to override the default location inside the source code repository where the build looks for the application source code. If your application exists inside a sub-directory, you can override the default location (the root folder) using this field.

When using the Git repository as a source without specifying the ref field, OpenShift Online performs a shallow clone (--depth=1 clone). That means only the HEAD (usually the master branch) is downloaded. This results in repositories downloading faster, including the commit history.

A shallow clone is also used when the ref field is specified and set to an existing remote branch name. However, if you specify the ref field to a specific commit, the system will fallback to a git clone operation and checkout the commit, because using the --depth=1 option only works with named branch refs.

If the ref field denotes a pull request, the system will use a git fetch operation and then checkout FETCH_HEAD.

To perform a full git clone of the master for the specified repository, set the ref to master.

8.3.3.1. Using a Proxy

If your Git repository can only be accessed using a proxy, you can define the proxy to use in the source section of the BuildConfig. You can configure both a HTTP and HTTPS proxy to use. Both fields are optional. Domains for which no proxying should be performed can also be specified via the NoProxy field.

Note

Your source URI must use the HTTP or HTTPS protocol for this to work.

source:
  git:
    uri: "https://github.com/openshift/ruby-hello-world"
    httpProxy: http://proxy.example.com
    httpsProxy: https://proxy.example.com
    noProxy: somedomain.com, otherdomain.com
Note

For Pipeline strategy builds, given the current restrictions with the Git plug-in for Jenkins, any Git operations through the Git plug-in will not leverage the HTTP or HTTPS proxy defined in the BuildConfig. The Git plug-in only will use the the proxy configured in the Jenkins UI at the Plugin Manager panel. This proxy will then be used for all git interactions within Jenkins, across all jobs. You can find instructions on how to configure proxies through the Jenkins UI at JenkinsBehindProxy.

8.3.3.2. Source Clone Secrets

Builder pods require access to any Git repositories defined as source for a build. Source clone secrets are used to provide the builder pod with access it would not normally have access to, such as private repositories or repositories with self-signed or untrusted SSL certificates.

The following source clone secret configurations are supported.

Note

You can also use combinations of these configurations to meet your specific needs.

Builds are run with the builder service account, which must have access to any source clone secrets used. Access is granted with the following command:

$ oc secrets link builder mysecret
Note

Limiting secrets to only the service accounts that reference them is disabled by default. This means that if serviceAccountConfig.limitSecretReferences is set to false (the default setting) in the master configuration file, linking secrets to a service is not required.

8.3.3.2.1. Automatically Adding a Source Clone Secret to a Build Configuration

When a BuildConfig is created, OpenShift Online can automatically populate its source clone secret reference. This behaviour allows the resulting Builds to automatically use the credentials stored in the referenced Secret to authenticate to a remote Git repository, without requiring further configuration.

To use this functionality, a Secret containing the Git repository credentials must exist in the namespace in which the BuildConfig will later be created. This Secret must additionally include one or more annotations prefixed with build.openshift.io/source-secret-match-uri-. The value of each of these annotations is a URI pattern, defined as follows. When a BuildConfig is created without a source clone secret reference and its Git source URI matches a URI pattern in a Secret annotation, OpenShift Online will automatically insert a reference to that Secret in the BuildConfig.

A URI pattern must consist of:

  • a valid scheme (*://, git://, http://, https:// or ssh://).
  • a host (* or a valid hostname or IP address optionally preceded by *.).
  • a path (/* or / followed by any characters optionally including * characters).

In all of the above, a * character is interpreted as a wildcard.

Note

URI patterns only match Git source URIs which are conformant to RFC3986. For example, https://github.com/openshift/origin.git. They do not match the alternate SSH style that Git also uses. For example, git@github.com:openshift/origin.git.

It is not valid to attempt to express a URI pattern in the alternate style, or to include a username/password component in a URI pattern.

If multiple Secrets match the Git URI of a particular BuildConfig, OpenShift Online will select the secret with the longest match. This allows for basic overriding, as in the following example.

The following fragment shows two partial source clone secrets, the first matching any server in the domain mycorp.com accessed by HTTPS, and the second overriding access to servers mydev1.mycorp.com and mydev2.mycorp.com:

kind: Secret
apiVersion: v1
metadata:
  name: matches-all-corporate-servers-https-only
  annotations:
    build.openshift.io/source-secret-match-uri-1: https://*.mycorp.com/*
data:
  ...

kind: Secret
apiVersion: v1
metadata:
  name: override-for-my-dev-servers-https-only
  annotations:
    build.openshift.io/source-secret-match-uri-1: https://mydev1.mycorp.com/*
    build.openshift.io/source-secret-match-uri-2: https://mydev2.mycorp.com/*
data:
  ...

Add a build.openshift.io/source-secret-match-uri- annotation to a pre-existing secret using:

$ oc annotate secret mysecret \
    'build.openshift.io/source-secret-match-uri-1=https://*.mycorp.com/*'
8.3.3.2.2. Manually Adding Source Clone Secrets

Source clone secrets can be added manually to a build configuration by adding a sourceSecret field to the source section inside the BuildConfig and setting it to the name of the secret that you created (basicsecret, in this example).

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "sample-build"
spec:
  output:
    to:
      kind: "ImageStreamTag"
      name: "sample-image:latest"
  source:
    git:
      uri: "https://github.com/user/app.git"
    sourceSecret:
      name: "basicsecret"
  strategy:
    sourceStrategy:
      from:
        kind: "ImageStreamTag"
        name: "python-33-centos7:latest"
Note

You can also use the oc set build-secret command to set the source clone secret on an existing build configuration:

$ oc set build-secret --source bc/sample-build basicsecret

Defining Secrets in the BuildConfig provides more information on this topic.

8.3.3.2.3. .Gitconfig File

If the cloning of your application is dependent on a .gitconfig file, then you can create a secret that contains it, and then add it to the builder service account, and then your BuildConfig.

To create a secret from a .gitconfig file:

$ oc secrets new mysecret .gitconfig=path/to/.gitconfig
Note

SSL verification can be turned off if sslVerify=false is set for the http section in your .gitconfig file:

[http]
        sslVerify=false
8.3.3.2.4. Basic Authentication

Basic authentication requires either a combination of --username and --password, or a token to authenticate against the SCM server.

Create the secret first before using the user name and password to access the private repository:

$ oc secrets new-basicauth <secret_name> \
    --username=<user_name> \
    --password=<password>

To create a basic authentication secret with a token:

$ oc secrets new-basicauth <secret_name> \
    --password=<token>
8.3.3.2.5. SSH Key Authentication

SSH key based authentication requires a private SSH key.

The repository keys are usually located in the $HOME/.ssh/ directory, and are named id_dsa.pub, id_ecdsa.pub, id_ed25519.pub, or id_rsa.pub by default. Generate SSH key credentials with the following command:

$ ssh-keygen -t rsa -C "your_email@example.com"
Note

Creating a passphrase for the SSH key prevents OpenShift Online from building. When prompted for a passphrase, leave it blank.

Two files are created: the public key and a corresponding private key (one of id_dsa, id_ecdsa, id_ed25519, or id_rsa). With both of these in place, consult your source control management (SCM) system’s manual on how to upload the public key. The private key is used to access your private repository.

Before using the SSH key to access the private repository, create the secret first:

$ oc secrets new-sshauth sshsecret \
    --ssh-privatekey=$HOME/.ssh/id_rsa
8.3.3.2.6. Trusted Certificate Authorities

The set of TLS certificate authorities that are trusted during a git clone operation are built into the OpenShift Online infrastructure images. If your Git server uses a self-signed certificate or one signed by an authority not trusted by the image, you have several options.

  1. Create a secret with a CA certificate file (recommended).

    A secret containing a CA certificate in a key named ca.crt will automatically be used by Git to trust your self-signed or otherwise un-trusted TLS certificate during the git clone operation. Using this method is significantly more secure than disabling Git’s SSL verification, which accepts any TLS certificate that is presented.

    $ oc secrets new mycert ca.crt=</path/to/file> 1
    1
    The key name ca.crt must be used.
  2. Disable Git TLS verification.

    You can disable Git’s TLS verification by setting the GIT_SSL_NO_VERIFY environment variable to true in the appropriate strategy section of your build configuration. You can use the oc set env command to manage BuildConfig environment variables.

8.3.3.2.7. Combinations

Below are several examples of how you can combine the above methods for creating source clone secrets for your specific needs.

  1. To create an SSH-based authentication secret with a .gitconfig file:

    $ oc secrets new-sshauth sshsecret \
        --ssh-privatekey=$HOME/.ssh/id_rsa \
        --gitconfig=</path/to/file>
  2. To create a secret that combines a .gitconfig file and CA certificate:

    $ oc secrets new mysecret \
        ca.crt=path/to/certificate \
        .gitconfig=path/to/.gitconfig
  3. To create a basic authentication secret with a CA certificate file:

    $ oc secrets new-basicauth <secret_name> \
        --username=<user_name> \
        --password=<password> \
        --ca-cert=</path/to/file>
  4. To create a basic authentication secret with a .gitconfig file:

    $ oc secrets new-basicauth <secret_name> \
        --username=<user_name> \
        --password=<password> \
        --gitconfig=</path/to/file>
  5. To create a basic authentication secret with a .gitconfig file and CA certificate file:

    $ oc secrets new-basicauth <secret_name> \
        --username=<user_name> \
        --password=<password> \
        --gitconfig=</path/to/file> \
        --ca-cert=</path/to/file>

8.3.4. Binary Source

Streaming content in binary format from a local file system to the builder is called a Binary type build. The corresponding value of BuildConfig.spec.source.type is Binary for such builds.

This source type is unique in that it is leveraged solely based on your use of the oc start-build.

Note

Binary type builds require content to be streamed from the local file system, so automatically triggering a binary type build (e.g. via an image change trigger) is not possible, because the binary files cannot be provided. Similarly, you cannot launch binary type builds from the web console.

To utilize binary builds, invoke oc start-build with one of these options:

  • --from-file: The contents of the file you specify are sent as a binary stream to the builder. You can also specify a URL to a file. Then, the builder stores the data in a file with the same name at the top of the build context.
  • --from-dir and --from-repo: The contents are archived and sent as a binary stream to the builder. Then, the builder extracts the contents of the archive within the build context directory. With --from-dir, you can also specify a URL to an archive, which will be extracted.
  • --from-archive: The archive you specify is sent to the builder, where it is extracted within the build context directory. This option behaves the same as --from-dir; an archive is created on your host first, whenever the argument to these options is a directory.

In each of the above cases:

  • If your BuildConfig already has a Binary source type defined, it will effectively be ignored and replaced by what the client sends.
  • If your BuildConfig has a Git source type defined, it is dynamically disabled, since Binary and Git are mutually exclusive, and the data in the binary stream provided to the builder takes precedence.

Instead of a file name, you can pass a URL with HTTP or HTTPS schema to --from-file and --from-archive. When using --from-file with a URL, the name of the file in the builder image is determined by the Content-Disposition header sent by the web server, or the last component of the URL path if the header is not present. No form of authentication is supported and it is not possible to use custom TLS certificate or disable certificate validation.

When using oc new-build --binary=true, the command ensures that the restrictions associated with binary builds are enforced. The resulting BuildConfig will have a source type of Binary, meaning that the only valid way to run a build for this BuildConfig is to use oc start-build with one of the --from options to provide the requisite binary data.

In the case of the binary stream encapsulating extracted archive content, the value of the contextDir field is interpreted as a subdirectory within the archive, and, if valid, the builder changes into that subdirectory before executing the build.

8.3.5. Input Secrets

In some scenarios, build operations require credentials to access dependent resources, but it is undesirable for those credentials to be available in the final application image produced by the build. You can define input secrets for this purpose.

For example, when building a Node.js application, you can set up your private mirror for Node.js modules. In order to download modules from that private mirror, you have to supply a custom .npmrc file for the build that contains a URL, user name, and password. For security reasons, you do not want to expose your credentials in the application image.

This example describes Node.js, but you can use the same approach for adding SSL certificates into the /etc/ssl/certs directory, API keys or tokens, license files, and more.

8.3.5.1. Adding Input Secrets

To add an input secret to an existing BuildConfig:

  1. Create the secret, if it does not exist:

    $ oc secrets new secret-npmrc .npmrc=~/.npmrc

    This creates a new secret named secret-npmrc, which contains the base64 encoded content of the ~/.npmrc file.

  2. Add the secret to the source section in the existing BuildConfig:

    source:
      git:
        uri: https://github.com/openshift/nodejs-ex.git
      secrets:
        - secret:
            name: secret-npmrc

To include the secret in a new BuildConfig, run the following command:

$ oc new-build \
    openshift/nodejs-010-centos7~https://github.com/openshift/nodejs-ex.git \
    --build-secret secret-npmrc

During the build, the .npmrc file is copied into the directory where the source code is located. In OpenShift Online S2I builder images, this is the image working directory, which is set using the WORKDIR instruction in the Dockerfile. If you want to specify another directory, add a destinationDir to the secret definition:

source:
  git:
    uri: https://github.com/openshift/nodejs-ex.git
  secrets:
    - secret:
        name: secret-npmrc
      destinationDir: /etc

You can also specify the destination directory when creating a new BuildConfig:

$ oc new-build \
    openshift/nodejs-010-centos7~https://github.com/openshift/nodejs-ex.git \
    --build-secret “secret-npmrc:/etc”

In both cases, the .npmrc file is added to the /etc directory of the build environment.

8.3.5.2. Source-to-Image Strategy

When using a Source strategy, all defined input secrets are copied to their respective destinationDir. If you left destinationDir empty, then the secrets are placed in the working directory of the builder image.

The same rule is used when a destinationDir is a relative path; the secrets are placed in the paths that are relative to the image’s working directory. The destinationDir must exist or an error will occur. No directory paths are created during the copy process.

Note

Currently, any files with these secrets are world-writable (have 0666 permissions) and will be truncated to size zero after executing the assemble script. This means that the secret files will exist in the resulting image, but they will be empty for security reasons.

8.3.6. Using External Artifacts

It is not recommended to store binary files in a source repository. Therefore, you may find it necessary to define a build which pulls additional files (such as Java .jar dependencies) during the build process. How this is done depends on the build strategy you are using.

For a Source build strategy, you must put appropriate shell commands into the assemble script:

.s2i/bin/assemble File

#!/bin/sh
APP_VERSION=1.0
wget http://repository.example.com/app/app-$APP_VERSION.jar -O app.jar

.s2i/bin/run File

#!/bin/sh
exec java -jar app.jar

Note

For more information on how to control which assemble and run script is used by a Source build, see Overriding Builder Image Scripts.

In practice, you may want to use an environment variable for the file location so that the specific file to be downloaded can be customized using an environment variable defined on the BuildConfig, rather than updating the assemble script.

You can choose between different methods of defining environment variables:

8.3.7. Using Docker Credentials for Private Registries

You can supply builds with a .docker/config.json file with valid credentials for private Docker registries. This allows you to push the output image into a private Docker registry or pull a builder image from the private Docker registry that requires authentication.

Note

For the OpenShift Online Docker registry, this is not required because secrets are generated automatically for you by OpenShift Online.

The .docker/config.json file is found in your home directory by default and has the following format:

auths:
  https://index.docker.io/v1/: 1
    auth: "YWRfbGzhcGU6R2labnRib21ifTE=" 2
    email: "user@example.com" 3
1
URL of the registry.
2
Encrypted password.
3
Email address for the login.

You can define multiple Docker registry entries in this file. Alternatively, you can also add authentication entries to this file by running the docker login command. The file will be created if it does not exist.

Kubernetes provides Secret objects, which can be used to store configuration and passwords.

  1. Create the secret from your local .docker/config.json file:

    $ oc secrets new dockerhub ~/.docker/config.json

    This generates a JSON specification of the secret named dockerhub and creates the object.

  2. Once the secret is created, add it to the builder service account. Each build is run with the builder role, so you must give it access your secret with the following command:

    $ oc secrets link builder dockerhub
  3. Add a pushSecret field into the output section of the BuildConfig and set it to the name of the secret that you created, which in the above example is dockerhub:

    spec:
      output:
        to:
          kind: "DockerImage"
          name: "private.registry.com/org/private-image:latest"
        pushSecret:
          name: "dockerhub"

    You can also use the oc set build-secret command to set the push secret on the build configuration:

    $ oc set build-secret --push bc/sample-build dockerhub
  4. Pull the builder container image from a private Docker registry by specifying the pullSecret field, which is part of the build strategy definition:

    strategy:
      sourceStrategy:
        from:
          kind: "DockerImage"
          name: "docker.io/user/private_repository"
        pullSecret:
          name: "dockerhub"

    You can also use the oc set build-secret command to set the pull secret on the build configuration:

    $ oc set build-secret --pull bc/sample-build dockerhub

8.4. Build Output

8.4.1. Build Output Overview

Builds that use the Source strategy result in the creation of a new container image. The image is then pushed to the container image registry specified in the output section of the Build specification.

If the output kind is ImageStreamTag, then the image will be pushed to the integrated OpenShift Online registry and tagged in the specified image stream. If the output is of type DockerImage, then the name of the output reference will be used as a Docker push specification. The specification may contain a registry or will default to DockerHub if no registry is specified. If the output section of the build specification is empty, then the image will not be pushed at the end of the build.

Output to an ImageStreamTag

spec:
  output:
    to:
      kind: "ImageStreamTag"
      name: "sample-image:latest"

Output to a Docker Push Specification

spec:
  output:
    to:
      kind: "DockerImage"
      name: "my-registry.mycompany.com:5000/myimages/myimage:tag"

8.4.2. Output Image Environment Variables

Source strategy builds set the following environment variables on output images:

VariableDescription

OPENSHIFT_BUILD_NAME

Name of the build

OPENSHIFT_BUILD_NAMESPACE

Namespace of the build

OPENSHIFT_BUILD_SOURCE

The source URL of the build

OPENSHIFT_BUILD_REFERENCE

The Git reference used in the build

OPENSHIFT_BUILD_COMMIT

Source commit used in the build

Additionally, any user-defined environment variable, for example those configured via Source strategy options, will also be part of the output image environment variable list.

8.4.3. Output Image Labels

Source builds set the following labels on output images:

LabelDescription

io.openshift.build.commit.author

Author of the source commit used in the build

io.openshift.build.commit.date

Date of the source commit used in the build

io.openshift.build.commit.id

Hash of the source commit used in the build

io.openshift.build.commit.message

Message of the source commit used in the build

io.openshift.build.commit.ref

Branch or reference specified in the source

io.openshift.build.source-location

Source URL for the build

You can also use the BuildConfig.spec.output.imageLabels field to specify a list of custom labels that will be applied to each image built from the BuildConfig.

Custom Labels to be Applied to Built Images

spec:
  output:
    to:
      kind: "ImageStreamTag"
      name: "my-image:latest"
    imageLabels:
    - name: "vendor"
      value: "MyCompany"
    - name: "authoritative-source-url"
      value: "registry.mycompany.com"

8.4.4. Output Image Digest

Built images can be uniquely identified by their digest, which can later be used to pull the image by digest regardless of its current tag.

Source builds store the digest in Build.status.output.to.imageDigest after the image is pushed to a registry. The digest is computed by the registry. Therefore, it may not always be present, for example when the registry did not return a digest, or when the builder image did not understand its format.

Built Image Digest After a Successful Push to the Registry

status:
  output:
    to:
      imageDigest: sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912

8.4.5. Using Docker Credentials for Private Registries

To push an image to a private Docker registry, credentials can be supplied using a secret. See Build Inputs for instructions.

8.5. Build Strategy Options

Important

The Docker build strategy is not supported in OpenShift Online.

8.5.1. Source-to-Image Strategy Options

The following options are specific to the S2I build strategy.

8.5.1.1. Force Pull

By default, if the builder image specified in the build configuration is available locally on the node, that image will be used. However, to override the local image and refresh it from the registry to which the image stream points, create a BuildConfig with the forcePull flag set to true:

strategy:
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest" 1
    forcePull: true 2
1
The builder image being used, where the local version on the node may not be up to date with the version in the registry to which the image stream points.
2
This flag causes the local builder image to be ignored and a fresh version to be pulled from the registry to which the image stream points. Setting forcePull to false results in the default behavior of honoring the image stored locally.

8.5.1.2. Incremental Builds

S2I can perform incremental builds, which means it reuses artifacts from previously-built images. To create an incremental build, create a BuildConfig with the following modification to the strategy definition:

strategy:
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "incremental-image:latest" 1
    incremental: true 2
1
Specify an image that supports incremental builds. Consult the documentation of the builder image to determine if it supports this behavior.
2
This flag controls whether an incremental build is attempted. If the builder image does not support incremental builds, the build will still succeed, but you will get a log message stating the incremental build was not successful because of a missing save-artifacts script.
Note

See the S2I Requirements topic for information on how to create a builder image supporting incremental builds.

8.5.1.3. Overriding Builder Image Scripts

You can override the assemble, run, and save-artifactsS2I scripts provided by the builder image in one of two ways. Either:

  1. Provide an assemble, run, and/or save-artifacts script in the .s2i/bin directory of your application source repository, or
  2. Provide a URL of a directory containing the scripts as part of the strategy definition. For example:
strategy:
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest"
    scripts: "http://somehost.com/scripts_directory" 1
1
This path will have run, assemble, and save-artifacts appended to it. If any or all scripts are found they will be used in place of the same named script(s) provided in the image.
Note

Files located at the scripts URL take precedence over files located in .s2i/bin of the source repository. See the S2I Requirements topic and the S2I documentation for information on how S2I scripts are used.

8.5.1.4. Environment Variables

There are two ways to make environment variables available to the source build process and resulting image. Environment files and BuildConfig environment values. Variables provided will be present during the build process and in the output image.

8.5.1.4.1. Environment Files

Source build enables you to set environment values (one per line) inside your application, by specifying them in a .s2i/environment file in the source repository. The environment variables specified in this file are present during the build process and in the output image. The complete list of supported environment variables is available in the documentation for each image.

If you provide a .s2i/environment file in your source repository, S2I reads this file during the build. This allows customization of the build behavior as the assemble script may use these variables.

For example, if you want to disable assets compilation for your Rails application, you can add DISABLE_ASSET_COMPILATION=true in the .s2i/environment file to cause assets compilation to be skipped during the build.

In addition to builds, the specified environment variables are also available in the running application itself. For example, you can add RAILS_ENV=development to the .s2i/environment file to cause the Rails application to start in development mode instead of production.

8.5.1.4.2. BuildConfig Environment

You can add environment variables to the sourceStrategy definition of the BuildConfig. The environment variables defined there are visible during the assemble script execution and will be defined in the output image, making them also available to the run script and application code.

For example disabling assets compilation for your Rails application:

sourceStrategy:
...
  env:
    - name: "DISABLE_ASSET_COMPILATION"
      value: "true"

The Build Environment section provides more advanced instructions.

You can also manage environment variables defined in the BuildConfig with the oc set env command.

8.5.1.5. Adding Secrets via Web Console

To add a secret to your build configuration so that it can access a private repository:

  1. Create a new OpenShift Online project.
  2. Create a secret that contains credentials for accessing a private source code repository.
  3. Create a Source-to-Image (S2I) build configuration.
  4. On the build configuration editor page or in the create app from builder image page of the web console, set the Source Secret.
  5. Click the Save button.
8.5.1.5.1. Enabling Pulling and Pushing

Enable pulling to a private registry by setting the Pull Secret in the build configuration and enable pushing by setting the Push Secret.

8.5.1.6. Ignoring Source Files

Source to image supports a .s2iignore file, which contains a list of file patterns that should be ignored. Files in the build working directory, as provided by the various input sources, that match a pattern found in the .s2iignore file will not be made available to the assemble script.

For more details on the format of the .s2iignore file, see the source-to-image documentation.

8.5.2. Pipeline Strategy Options

The following options are specific to the Pipeline build strategy.

8.5.2.1. Providing the Jenkinsfile

You can provide the Jenkinsfile in one of two ways:

  1. Embed the Jenkinsfile in the build configuration.
  2. Include in the build configuration a reference to the Git repository that contains the Jenkinsfile.

Embedded Definition

kind: "BuildConfig"
apiVersion: "v1"
metadata:
  name: "sample-pipeline"
spec:
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfile: "node('agent') {\nstage 'build'\nopenshiftBuild(buildConfig: 'ruby-sample-build', showBuildLogs: 'true')\nstage 'deploy'\nopenshiftDeploy(deploymentConfig: 'frontend')\n}"

Reference to Git Repository

kind: "BuildConfig"
apiVersion: "v1"
metadata:
  name: "sample-pipeline"
spec:
  source:
    git:
      uri: "https://github.com/openshift/ruby-hello-world"
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: some/repo/dir/filename 1

1
The optional jenkinsfilePath field specifies the name of the file to use, relative to the source contextDir. If contextDir is omitted, it defaults to the root of the repository. If jenkinsfilePath is omitted, it defaults to Jenkinsfile.

8.5.2.2. Environment Variables

To make environment variables available to the Pipeline build process, you can add environment variables to the jenkinsPipelineStrategy definition of the BuildConfig.

Once defined, the environment variables will be set as parameters for any Jenkins job associated with the BuildConfig.

For example:

jenkinsPipelineStrategy:
...
  env:
    - name: "FOO"
      value: "BAR"
Note

You can also manage environment variables defined in the BuildConfig with the oc set env command.

8.5.2.2.1. Mapping Between BuildConfig Environment Variables and Jenkins Job Parameters

When a Jenkins job is created or updated based on changes to a Pipeline strategy BuildConfig, any environment variables in the BuildConfig are mapped to Jenkins job parameters definitions, where the default values for the Jenkins job parameters definitions are the current values of the associated environment variables.

After the Jenkins job’s initial creation, you can still add additional parameters to the job from the Jenkins console. The parameter names differ from the names of the environment variables in the BuildConfig. The parameters are honored when builds are started for those Jenkins jobs.

How you start builds for the Jenkins job dictates how the parameters are set. If you start with oc start-build, the values of the environment variables in the BuildConfig are the parameters set for the corresponding job instance. Any changes you make to the parameters' default values from the Jenkins console are ignored. The BuildConfig values take precedence.

If you start with oc start-build -e, the values for the environment variables specified in the -e option take precedence. And if you specify an environment variable not listed in the BuildConfig, they will be added as a Jenkins job parameter definitions. Also any changes you make from the Jenkins console to the parameters corresponding to the environment variables are ignored. The BuildConfig and what you specify with oc start-build -e takes precedence.

If you start the Jenkins job via the Jenkins console, then you can control the setting of the parameters via the Jenkins console as part of starting a build for the job.

8.6. Build Environment

8.6.1. Overview

As with pod environment variables, build environment variables can be defined in terms of references to other resources/variables using the Downward API. However, there are some exceptions as noted below.

Note

You can also manage environment variables defined in the BuildConfig with the oc set env command.

8.6.2. Using Build Fields as Environment Variables

You can inject information about the build object by setting the fieldPath environment variable source to the JsonPath of the field from which you are interested in obtaining the value.

env:
  - name: FIELDREF_ENV
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
Note

Jenkins Pipeline strategy does not support valueFrom syntax for environment variables.

8.6.3. Using Container Resources as Environment Variables

Referencing container resources using valueFrom in build environment variables is not supported as the references are resolved before the container is created.

8.7. Triggering Builds

8.7.1. Build Triggers Overview

When defining a BuildConfig, you can define triggers to control the circumstances in which the BuildConfig should be run. The following build triggers are available:

8.7.2. Webhook Triggers

Webhook triggers allow you to trigger a new build by sending a request to the OpenShift Online API endpoint. You can define these triggers using GitHub, GitLab, Bitbucket, or Generic webhooks.

OpenShift Online webhooks currently only support their analogous versions of the push event for each of the Git based source code management systems (SCMs). All other event types are ignored.

When the push events are processed, a confirmation is made as to whether the branch reference inside the event matches the branch reference in the corresponding BuildConfig. If they match, then the exact commit reference noted in the webhook event is checked out for the OpenShift Online build. If they do not match, no build is triggered.

Note

oc new-app and oc new-build will create GitHub and Generic webhook triggers automatically, but any other needed webhook triggers must be added manually (see Setting Triggers).

8.7.2.1. GitHub Webhooks

GitHub webhooks handle the call made by GitHub when a repository is updated. When defining the trigger, you must specify a secret, which will be part of the URL you supply to GitHub when configuring the webhook. The secret ensures the uniqueness of the URL, preventing others from triggering the build. The following example is a trigger definition YAML within the BuildConfig:

type: "GitHub"
github:
  secret: "secret101"
Note

The secret field in webhook trigger configuration is not the same as secret field you encounter when configuring webhook in GitHub UI. The former is to make the webhook URL unique and hard to predict, the latter is an optional string field used to create HMAC hex digest of the body, which is sent as an X-Hub-Signatureheader.

The payload URL is returned as the GitHub Webhook URL by the oc describe command (see Displaying Webhook URLs), and is structured as follows:

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/github

To configure a GitHub Webhook:

  1. Describe the build configuration to get the webhook URL:

    $ oc describe bc <name>
  2. Copy the webhook URL.
  3. Follow the GitHub setup instructions to paste the webhook URL into your GitHub repository settings.
Note

Gogs supports the same webhook payload format as GitHub. Therefore, if you are using a Gogs server, you can define a GitHub webhook trigger on your BuildConfig and trigger it via your Gogs server also.

Given a file containing a valid JSON payload, such as payload.json, you can manually trigger the webhook via curl:

$ curl -H "X-GitHub-Event: push" -H "Content-Type: application/json" -k -X POST --data-binary @payload.json https://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/github

The -k argument is only necessary if your API server does not have a properly signed certificate.

8.7.2.2. GitLab Webhooks

GitLab webhooks handle the call made by GitLab when a repository is updated. As with the GitHub triggers, you must specify a secret. The following example is a trigger definition YAML within the BuildConfig:

type: "GitLab"
gitlab:
  secret: "secret101"

The payload URL is returned as the GitLab Webhook URL by the oc describe command (see Displaying Webhook URLs), and is structured as follows:

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/gitlab

To configure a GitLab Webhook:

  1. Describe the build configuration to get the webhook URL:

    $ oc describe bc <name>
  2. Copy the webhook URL.
  3. Follow the GitLab setup instructions to paste the webhook URL into your GitLab repository settings.

Given a file containing a valid JSON payload, such as payload.json, you can manually trigger the webhook via curl:

$ curl -H "X-GitLab-Event: Push Hook" -H "Content-Type: application/json" -k -X POST --data-binary @payload.json https://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/gitlab

The -k argument is only necessary if your API server does not have a properly signed certificate.

8.7.2.3. Bitbucket Webhooks

Bitbucket webhooks handle the call made by Bitbucket when a repository is updated. Similar to the previous triggers, you must specify a secret. The following example is a trigger definition YAML within the BuildConfig:

type: "Bitbucket"
bitbucket:
  secret: "secret101"

The payload URL is returned as the Bitbucket Webhook URL by the oc describe command (see Displaying Webhook URLs), and is structured as follows:

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/bitbucket

To configure a Bitbucket Webhook:

  1. Describe the build configuration to get the webhook URL:

    $ oc describe bc <name>
  2. Copy the webhook URL.
  3. Follow the Bitbucket setup instructions to paste the webhook URL into your Bitbucket repository settings.

Given a file containing a valid JSON payload, such as payload.json, you can manually trigger the webhook via curl:

$ curl -H "X-Event-Key: repo:push" -H "Content-Type: application/json" -k -X POST --data-binary @payload.json https://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/bitbucket

The -k argument is only necessary if your API server does not have a properly signed certificate.

8.7.2.4. Generic Webhooks

Generic webhooks are invoked from any system capable of making a web request. As with the other webhooks, you must specify a secret, which will be part of the URL that the caller must use to trigger the build. The secret ensures the uniqueness of the URL, preventing others from triggering the build. The following is an example trigger definition YAML within the BuildConfig:

type: "Generic"
generic:
  secret: "secret101"
  allowEnv: true 1
1
Set to true to allow a generic webhook to pass in environment variables.

To set up the caller, supply the calling system with the URL of the generic webhook endpoint for your build:

http://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/generic

The caller must invoke the webhook as a POST operation.

To invoke the webhook manually you can use curl:

$ curl -X POST -k https://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/generic

The HTTP verb must be set to POST. The insecure -k flag is specified to ignore certificate validation. This second flag is not necessary if your cluster has properly signed certificates.

The endpoint can accept an optional payload with the following format:

git:
  uri: "<url to git repository>"
  ref: "<optional git reference>"
  commit: "<commit hash identifying a specific git commit>"
  author:
    name: "<author name>"
    email: "<author e-mail>"
  committer:
    name: "<committer name>"
    email: "<committer e-mail>"
  message: "<commit message>"
env: 1
   - name: "<variable name>"
     value: "<variable value>"
1
Similar to the BuildConfig environment variables, the environment variables defined here are made available to your build. If these variables collide with the BuildConfig environment variables, these variables take precedence. By default, environment variables passed via webhook are ignored. Set the allowEnv field to true on the webhook definition to enable this behavior.

To pass this payload using curl, define it in a file named payload_file.yaml and run:

$ curl -H "Content-Type: application/yaml" --data-binary @payload_file.yaml -X POST -k https://<openshift_api_host:port>/oapi/v1/namespaces/<namespace>/buildconfigs/<name>/webhooks/<secret>/generic

The arguments are the same as the previous example with the addition of a header and a payload. The -H argument sets the Content-Type header to application/yaml or application/json depending on your payload format. The --data-binary argument is used to send a binary payload with newlines intact with the POST request.

Note

OpenShift Online permits builds to be triggered via the generic webhook even if an invalid request payload is presented (for example, invalid content type, unparsable or invalid content, and so on). This behavior is maintained for backwards compatibility. If an invalid request payload is presented, OpenShift Online returns a warning in JSON format as part of its HTTP 200 OK response.

8.7.2.5. Displaying Webhook URLs

Use the following command to display any webhook URLs associated with a build configuration:

$ oc describe bc <name>

If the above command does not display any webhook URLs, then no webhook trigger is defined for that build configuration. See Setting Triggers to manually add triggers.

8.7.3. Image Change Triggers

Image change triggers allow your build to be automatically invoked when a new version of an upstream image is available. For example, if a build is based on top of a RHEL image, then you can trigger that build to run any time the RHEL image changes. As a result, the application image is always running on the latest RHEL base image.

Configuring an image change trigger requires the following actions:

  1. Define an ImageStream that points to the upstream image you want to trigger on:

    kind: "ImageStream"
    apiVersion: "v1"
    metadata:
      name: "ruby-20-centos7"

    This defines the image stream that is tied to a container image repository located at <system-registry>/<namespace>/ruby-20-centos7. The <system-registry> is defined as a service with the name docker-registry running in OpenShift Online.

  2. If an image stream is the base image for the build, set the from field in the build strategy to point to the image stream:

    strategy:
      sourceStrategy:
        from:
          kind: "ImageStreamTag"
          name: "ruby-20-centos7:latest"

    In this case, the sourceStrategy definition is consuming the latest tag of the image stream named ruby-20-centos7 located within this namespace.

  3. Define a build with one or more triggers that point to image streams:

    type: "imageChange" 1
    imageChange: {}
    type: "imagechange" 2
    imageChange:
      from:
        kind: "ImageStreamTag"
        name: "custom-image:latest"
    1
    An image change trigger that monitors the ImageStream and Tag as defined by the build strategy’s from field. The imageChange object here must be empty.
    2
    An image change trigger that monitors an arbitrary image stream. The imageChange part in this case must include a from field that references the ImageStreamTag to monitor.

When using an image change trigger for the strategy image stream, the generated build is supplied with an immutable Docker tag that points to the latest image corresponding to that tag. This new image reference will be used by the strategy when it executes for the build.

For other image change triggers that do not reference the strategy image stream, a new build will be started, but the build strategy will not be updated with a unique image reference.

In the example above that has an image change trigger for the strategy, the resulting build will be:

strategy:
  sourceStrategy:
    from:
      kind: "DockerImage"
      name: "172.30.17.3:5001/mynamespace/ruby-20-centos7:<immutableid>"

This ensures that the triggered build uses the new image that was just pushed to the repository, and the build can be re-run any time with the same inputs.

If a build is triggered due to a webhook trigger or manual request, the build that is created uses the <immutableid> resolved from the ImageStream referenced by the Strategy. This ensures that builds are performed using consistent image tags for ease of reproduction.

Note

Image streams that point to container images in v1 Docker registries only trigger a build once when the image stream tag becomes available and not on subsequent image updates. This is due to the lack of uniquely identifiable images in v1 Docker registries.

8.7.4. Configuration Change Triggers

A configuration change trigger allows a build to be automatically invoked as soon as a new BuildConfig is created. The following is an example trigger definition YAML within the BuildConfig:

  type: "ConfigChange"
Note

Configuration change triggers currently only work when creating a new BuildConfig. In a future release, configuration change triggers will also be able to launch a build whenever a BuildConfig is updated.

8.7.4.1. Setting Triggers Manually

Triggers can be added to and removed from build configurations with oc set triggers. For example, to set a GitHub webhook trigger on a build configuration, use:

$ oc set triggers bc <name> --from-github

To set an imagechange trigger, use

$ oc set triggers bc <name> --from-image='<image>'

To remove a trigger, add --remove:

$ oc set triggers bc <name> --from-bitbucket --remove
Note

When a webhook trigger already exists, adding it again regenerates the webhook secret.

For more information, consult the help documentation with oc set triggers --help

8.8. Build Hooks

8.8.1. Build Hooks Overview

Build hooks allow behavior to be injected into the build process.

The postCommit field of a BuildConfig object executes commands inside a temporary container that is running the build output image. The hook is executed immediately after the last layer of the image has been committed and before the image is pushed to a registry.

The current working directory is set to the image’s WORKDIR, which is the default working directory of the container image. For most images, this is where the source code is located.

The hook fails if the script or command returns a non-zero exit code or if starting the temporary container fails. When the hook fails it marks the build as failed and the image is not pushed to a registry. The reason for failing can be inspected by looking at the build logs.

Build hooks can be used to run unit tests to verify the image before the build is marked complete and the image is made available in a registry. If all tests pass and the test runner returns with exit code 0, the build is marked successful. In case of any test failure, the build is marked as failed. In all cases, the build log will contain the output of the test runner, which can be used to identify failed tests.

The postCommit hook is not only limited to running tests, but can be used for other commands as well. Since it runs in a temporary container, changes made by the hook do not persist, meaning that the hook execution cannot affect the final image. This behavior allows for, among other uses, the installation and usage of test dependencies that are automatically discarded and will be not present in the final image.

8.8.2. Configuring Post Commit Build Hooks

There are different ways to configure the post build hook. All forms in the following examples are equivalent and execute bundle exec rake test --verbose:

  • Shell script:

    postCommit:
      script: "bundle exec rake test --verbose"

    The script value is a shell script to be run with /bin/sh -ic. Use this when a shell script is appropriate to execute the build hook. For example, for running unit tests as above. To control the image entry point, or if the image does not have /bin/sh, use command and/or args.

    Note

    The additional -i flag was introduced to improve the experience working with CentOS and RHEL images, and may be removed in a future release.

  • Command as the image entry point:

    postCommit:
      command: ["/bin/bash", "-c", "bundle exec rake test --verbose"]

    In this form, command is the command to run, which overrides the image entry point in the exec form, as documented in the Dockerfile reference. This is needed if the image does not have /bin/sh, or if you do not want to use a shell. In all other cases, using script might be more convenient.

  • Pass arguments to the default entry point:

    postCommit:
      args: ["bundle", "exec", "rake", "test", "--verbose"]

    In this form, args is a list of arguments that are provided to the default entry point of the image. The image entry point must be able to handle arguments.

  • Shell script with arguments:

    postCommit:
      script: "bundle exec rake test $1"
      args: ["--verbose"]

    Use this form if you need to pass arguments that would otherwise be hard to quote properly in the shell script. In the script, $0 will be "/bin/sh" and $1, $2, etc, are the positional arguments from args.

  • Command with arguments:

    postCommit:
      command: ["bundle", "exec", "rake", "test"]
      args: ["--verbose"]

    This form is equivalent to appending the arguments to command.

Note

Providing both script and command simultaneously creates an invalid build hook.

8.8.2.1. Using the CLI

The oc set build-hook command can be used to set the build hook for a build configuration.

To set a command as the post-commit build hook:

$ oc set build-hook bc/mybc \
    --post-commit \
    --command \
    -- bundle exec rake test --verbose

To set a script as the post-commit build hook:

$ oc set build-hook bc/mybc --post-commit --script="bundle exec rake test --verbose"

8.9. Build Run Policy

8.9.1. Build Run Policy Overview

The build run policy describes the order in which the builds created from the build configuration should run. This can be done by changing the value of the runPolicy field in the spec section of the Build specification.

It is also possible to change the runPolicy value for existing build configurations.

  • Changing Parallel to Serial or SerialLatestOnly and triggering a new build from this configuration will cause the new build to wait until all parallel builds complete as the serial build can only run alone.
  • Changing Serial to SerialLatestOnly and triggering a new build will cause cancellation of all existing builds in queue, except the currently running build and the most recently created build. The newest build will execute next.

8.9.2. Serial Run Policy

Setting the runPolicy field to Serial will cause all new builds created from the Build configuration to be run sequentially. That means there will be only one build running at a time and every new build will wait until the previous build completes. Using this policy will result in consistent and predictable build output. This is the default runPolicy.

Triggering three builds from the sample-build configuration, using the Serial policy will result in:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Running   13 seconds ago   13s
sample-build-2   Source    Git           New
sample-build-3   Source    Git           New

When the sample-build-1 build completes, the sample-build-2 build will run:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Completed 43 seconds ago   34s
sample-build-2   Source    Git@1aa381b   Running   2 seconds ago    2s
sample-build-3   Source    Git           New

8.9.3. SerialLatestOnly Run Policy

Setting the runPolicy field to SerialLatestOnly will cause all new builds created from the Build configuration to be run sequentially, same as using the Serial run policy. The difference is that when a currently running build completes, the next build that will run is the latest build created. In other words, you do not wait for the queued builds to run, as they are skipped. Skipped builds are marked as Cancelled. This policy can be used for fast, iterative development.

Triggering three builds from the sample-build configuration, using the SerialLatestOnly policy will result in:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Running   13 seconds ago   13s
sample-build-2   Source    Git           Cancelled
sample-build-3   Source    Git           New

The sample-build-2 build will be canceled (skipped) and the next build run after sample-build-1 completes will be the sample-build-3 build:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Completed 43 seconds ago   34s
sample-build-2   Source    Git           Cancelled
sample-build-3   Source    Git@1aa381b   Running   2 seconds ago    2s

8.9.4. Parallel Run Policy

Setting the runPolicy field to Parallel causes all new builds created from the Build configuration to be run in parallel. This can produce unpredictable results, as the first created build can complete last, which will replace the pushed container image produced by the last build which completed earlier.

Use the parallel run policy in cases where you do not care about the order in which the builds will complete.

Triggering three builds from the sample-build configuration, using the Parallel policy will result in three simultaneous builds:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Running   13 seconds ago   13s
sample-build-2   Source    Git@a76d881   Running   15 seconds ago   3s
sample-build-3   Source    Git@689d111   Running   17 seconds ago   3s

The completion order is not guaranteed:

NAME             TYPE      FROM          STATUS    STARTED          DURATION
sample-build-1   Source    Git@e79d887   Running   13 seconds ago   13s
sample-build-2   Source    Git@a76d881   Running   15 seconds ago   3s
sample-build-3   Source    Git@689d111   Completed 17 seconds ago   5s

8.10. Advanced Build Operations

8.10.1. Setting Build Resources

By default, builds are completed by pods using unbound resources, such as memory and CPU. These resources can be limited by specifying resource limits in a project’s default container limits.

You can also limit resource use by specifying resource limits as part of the build configuration. In the following example, each of the resources, cpu, and memory parameters are optional:

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "sample-build"
spec:
  resources:
    limits:
      cpu: "100m" 1
      memory: "256Mi" 2
1
cpu is in CPU units: 100m represents 0.1 CPU units (100 * 1e-3).
2
memory is in bytes: 256Mi represents 268435456 bytes (256 * 2 ^ 20).

However, if a quota has been defined for your project, one of the following two items is required:

  • A resources section set with an explicit requests:

    resources:
      requests: 1
        cpu: "100m"
        memory: "256Mi"
    1
    The requests object contains the list of resources that correspond to the list of resources in the quota.
  • A limit range defined in your project, where the defaults from the LimitRange object apply to pods created during the build process.

Otherwise, build pod creation will fail, citing a failure to satisfy quota.

8.10.2. Setting Maximum Duration

When defining a BuildConfig, you can define its maximum duration by setting the completionDeadlineSeconds field. It is specified in seconds and is not set by default. When not set, there is no maximum duration enforced.

The maximum duration is counted from the time when a build pod gets scheduled in the system, and defines how long it can be active, including the time needed to pull the builder image. After reaching the specified timeout, the build is terminated by OpenShift Online.

The following example shows the part of a BuildConfig specifying completionDeadlineSeconds field for 30 minutes:

spec:
  completionDeadlineSeconds: 1800

8.10.3. Assigning Builds to Specific Nodes

Builds can be targeted to run on specific nodes by specifying labels in the nodeSelector field of a build configuration. The nodeSelector value is a set of key/value pairs that are matched to node labels when scheduling the build pod.

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "sample-build"
spec:
  nodeSelector:1
    key1: value1
    key2: value2
1
Builds associated with this build configuration will run only on nodes with the key1=value2 and key2=value2 labels.

The nodeSelector value can also be controlled by cluster-wide default and override values. Defaults will only be applied if the build configuration does not define any key/value pairs for the nodeSelector and also does not define an explicitly empty map value of nodeSelector:{}. Override values will replace values in the build configuration on a key by key basis.

Note

If the specified NodeSelector cannot be matched to a node with those labels, the build still stay in the Pending state indefinitely.

8.10.4. Chaining Builds

For compiled languages (Go, C, C++, Java, etc.), including the dependencies necessary for compilation in the application image might increase the size of the image or introduce vulnerabilities that can be exploited.

To avoid these problems, two builds can be chained together: one that produces the compiled artifact, and a second build that places that artifact in a separate image that runs the artifact.

8.10.5. Build Pruning

By default, builds that have completed their lifecycle are persisted indefinitely. You can limit the number of previous builds that are retained by supplying a positive integer value for successfulBuildsHistoryLimit or failedBuildsHistoryLimit as shown in the following sample build configuration.

apiVersion: "v1"
kind: "BuildConfig"
metadata:
  name: "sample-build"
spec:
  successfulBuildsHistoryLimit: 2 1
  failedBuildsHistoryLimit: 2 2
1
successfulBuildsHistoryLimit will retain up to two builds with a status of completed.
2
failedBuildsHistoryLimit will retain up to two builds with a status of failed, cancelled, or error.

Build pruning is triggered by the following actions:

  • Updating a build configuration.
  • A build completes its lifecycle.

Builds are sorted by their creation timestamp with the oldest builds being pruned first.

8.11. Build Troubleshooting

8.11.1. Requested Access to Resources Denied

Issue

A build fails with:

requested access to the resource is denied
Resolution

You have exceeded one of the image quotas set on your project. Check your current quota and verify the limits applied and storage in use:

$ oc describe quota

Chapter 9. Deployments

9.1. How Deployments Work

9.1.1. What Is a Deployment?

OpenShift Online deployments provide fine-grained management over common user applications. They are described using three separate API objects:

  • A deployment configuration, which describes the desired state of a particular component of the application as a pod template.
  • One or more replication controllers, which contain a point-in-time record of the state of a deployment configuration as a pod template.
  • One or more pods, which represent an instance of a particular version of an application.
Important

Users do not need to manipulate replication controllers or pods owned by deployment configurations. The deployment system ensures changes to deployment configurations are propagated appropriately. If the existing deployment strategies are not suited for your use case and you have the need to run manual steps during the lifecycle of your deployment, then you should consider creating a custom strategy.

When you create a deployment configuration, a replication controller is created representing the deployment configuration’s pod template. If the deployment configuration changes, a new replication controller is created with the latest pod template, and a deployment process runs to scale down the old replication controller and scale up the new replication controller.

Instances of your application are automatically added and removed from both service load balancers and routers as they are created. As long as your application supports graceful shutdown when it receives the TERM signal, you can ensure that running user connections are given a chance to complete normally.

Features provided by the deployment system:

  • A deployment configuration, which is a template for running applications.
  • Triggers that drive automated deployments in response to events.
  • User-customizable strategies to transition from the previous version to the new version. A strategy runs inside a pod commonly referred as the deployment process.
  • A set of hooks for executing custom behavior in different points during the lifecycle of a deployment.
  • Versioning of your application in order to support rollbacks either manually or automatically in case of deployment failure.
  • Manual replication scaling and autoscaling.

9.1.2. Creating a Deployment Configuration

Deployment configurations are deploymentConfig OpenShift Online API resources which can be managed with the oc command like any other resource. The following is an example of a deploymentConfig resource:

kind: "DeploymentConfig"
apiVersion: "v1"
metadata:
  name: "frontend"
spec:
  template: 1
    metadata:
      labels:
        name: "frontend"
    spec:
      containers:
        - name: "helloworld"
          image: "openshift/origin-ruby-sample"
          ports:
            - containerPort: 8080
              protocol: "TCP"
  replicas: 5 2
  triggers:
    - type: "ConfigChange" 3
    - type: "ImageChange" 4
      imageChangeParams:
        automatic: true
        containerNames:
          - "helloworld"
        from:
          kind: "ImageStreamTag"
          name: "origin-ruby-sample:latest"
  strategy: 5
    type: "Rolling"
  paused: false 6
  revisionHistoryLimit: 2 7
  minReadySeconds: 0 8
1
The pod template of the frontend deployment configuration describes a simple Ruby application.
2
There will be 5 replicas of frontend.
3
A configuration change trigger causes a new replication controller to be created any time the pod template changes.
4
An image change trigger trigger causes a new replication controller to be created each time a new version of the origin-ruby-sample:latest image stream tag is available.
5
The Rolling strategy is the default way of deploying your pods. May be omitted.
6
Pause a deployment configuration. This disables the functionality of all triggers and allows for multiple changes on the pod template before actually rolling it out.
7
Revision history limit is the limit of old replication controllers you want to keep around for rolling back. May be omitted. If omitted, old replication controllers will not be cleaned up.
8
Minimum seconds to wait (after the readiness checks succeed) for a pod to be considered available. The default value is 0.

9.2. Basic Deployment Operations

9.2.1. Starting a Deployment

You can start a new deployment process manually using the web console, or from the CLI:

$ oc rollout latest dc/<name>
Note

If a deployment process is already in progress, the command will display a message and a new replication controller will not be deployed.

9.2.2. Viewing a Deployment

To get basic information about all the available revisions of your application:

$ oc rollout history dc/<name>

This will show details about all recently created replication controllers for the provided deployment configuration, including any currently running deployment process.

You can view details specific to a revision by using the --revision flag:

$ oc rollout history dc/<name> --revision=1

For more detailed information about a deployment configuration and its latest revision:

$ oc describe dc <name>
Note

The web console shows deployments in the Browse tab.

9.2.3. Canceling a Deployment

To cancel a running or stuck deployment process:

$ oc deploy --cancel dc/<name>
Warning

The cancellation is a best-effort operation, and may take some time to complete. The replication controller may partially or totally complete its deployment before the cancellation is effective. When canceled, the deployment configuration will be automatically rolled back by scaling up the previous running replication controller.

9.2.4. Retrying a Deployment

If the current revision of your deployment configuration failed to deploy, you can restart the deployment process with:

$ oc deploy --retry dc/<name>

If the latest revision of it was deployed successfully, the command will display a message and the deployment process will not be retried.

Note

Retrying a deployment restarts the deployment process and does not create a new deployment revision. The restarted replication controller will have the same configuration it had when it failed.

9.2.5. Rolling Back a Deployment

Rollbacks revert an application back to a previous revision and can be performed using the REST API, the CLI, or the web console.

To rollback to the last successful deployed revision of your configuration:

$ oc rollout undo dc/<name>

The deployment configuration’s template will be reverted to match the deployment revision specified in the undo command, and a new replication controller will be started. If no revision is specified with --to-revision, then the last successfully deployed revision will be used.

Image change triggers on the deployment configuration are disabled as part of the rollback to prevent accidentally starting a new deployment process soon after the rollback is complete. To re-enable the image change triggers:

$ oc set triggers dc/<name> --auto
Note

Deployment configurations also support automatically rolling back to the last successful revision of the configuration in case the latest deployment process fails. In that case, the latest template that failed to deploy stays intact by the system and it is up to users to fix their configurations.

9.2.6. Executing Commands Inside a Container

You can add a command to a container, which modifies the container’s startup behavior by overruling the image’s ENTRYPOINT. This is different from a lifecycle hook, which instead can be run once per deployment at a specified time.

Add the command parameters to the spec field of the deployment configuration. You can also add an args field, which modifies the command (or the ENTRYPOINT if command does not exist).

...
spec:
  containers:
    -
    name: <container_name>
    image: 'image'
    command:
      - '<command>'
    args:
      - '<argument_1>'
      - '<argument_2>'
      - '<argument_3>'
...

For example, to execute the java command with the -jar and /opt/app-root/springboots2idemo.jar arguments:

...
spec:
  containers:
    -
    name: example-spring-boot
    image: 'image'
    command:
      - java
    args:
      - '-jar'
      - /opt/app-root/springboots2idemo.jar
...

9.2.7. Viewing Deployment Logs

To stream the logs of the latest revision for a given deployment configuration:

$ oc logs -f dc/<name>

If the latest revision is running or failed, oc logs will return the logs of the process that is responsible for deploying your pods. If it is successful, oc logs will return the logs from a pod of your application.

You can also view logs from older failed deployment processes, if and only if these processes (old replication controllers and their deployer pods) exist and have not been pruned or deleted manually:

$ oc logs --version=1 dc/<name>

For more options on retrieving logs see:

$ oc logs --help

9.2.8. Setting Deployment Triggers

A deployment configuration can contain triggers, which drive the creation of new deployment processes in response to events inside the cluster.

Warning

If no triggers are defined on a deployment configuration, a ConfigChange trigger is added by default. If triggers are defined as an empty field, deployments must be started manually.

9.2.8.1. Configuration Change Trigger

The ConfigChange trigger results in a new replication controller whenever changes are detected in the pod template of the deployment configuration.

Note

If a ConfigChange trigger is defined on a deployment configuration, the first replication controller will be automatically created soon after the deployment configuration itself is created and it is not paused.

Example 9.1. A ConfigChange Trigger

triggers:
  - type: "ConfigChange"

9.2.8.2. ImageChange Trigger

The ImageChange trigger results in a new replication controller whenever the content of an image stream tag changes (when a new version of the image is pushed).

Example 9.2. An ImageChange Trigger

triggers:
  - type: "ImageChange"
    imageChangeParams:
      automatic: true 1
      from:
        kind: "ImageStreamTag"
        name: "origin-ruby-sample:latest"
        namespace: "myproject"
      containerNames:
        - "helloworld"
1
If the imageChangeParams.automatic field is set to false, the trigger is disabled.

With the above example, when the latest tag value of the origin-ruby-sample image stream changes and the new image value differs from the current image specified in the deployment configuration’s helloworld container, a new replication controller is created using the new image for the helloworld container.

Note

If an ImageChange trigger is defined on a deployment configuration (with a ConfigChange trigger and automatic=false, or with automatic=true) and the ImageStreamTag pointed by the ImageChange trigger does not exist yet, then the initial deployment process will automatically start as soon as an image is imported or pushed by a build to the ImageStreamTag.

9.2.8.2.1. Using the Command Line

The oc set triggers command can be used to set a deployment trigger for a deployment configuration. For the example above, you can set the ImageChangeTrigger by using the following command:

$ oc set triggers dc/frontend --from-image=myproject/origin-ruby-sample:latest -c helloworld

For more information, see:

$ oc set triggers --help

9.2.9. Setting Deployment Resources

A deployment is completed by a pod that consumes resources (memory and CPU) on a node. By default, pods consume unbounded node resources. However, if a project specifies default container limits, then pods consume resources up to those limits.

You can also limit resource use by specifying resource limits as part of the deployment strategy. Deployment resources can be used with the Recreate, Rolling, or Custom deployment strategies.

In the following example, each of resources, cpu, and memory is optional:

type: "Recreate"
resources:
  limits:
    cpu: "100m" 1
    memory: "256Mi" 2
1
cpu is in CPU units: 100m represents 0.1 CPU units (100 * 1e-3).
2
memory is in bytes: 256Mi represents 268435456 bytes (256 * 2 ^ 20).

However, if a quota has been defined for your project, one of the following two items is required:

  • A resources section set with an explicit requests:

      type: "Recreate"
      resources:
        requests: 1
          cpu: "100m"
          memory: "256Mi"
    1
    The requests object contains the list of resources that correspond to the list of resources in the quota.
  • A limit range defined in your project, where the defaults from the LimitRange object apply to pods created during the deployment process.

Otherwise, deploy pod creation will fail, citing a failure to satisfy quota.

9.2.10. Manual Scaling

In addition to rollbacks, you can exercise fine-grained control over the number of replicas from the web console, or by using the oc scale command. For example, the following command sets the replicas in the deployment configuration frontend to 3.

$ oc scale dc frontend --replicas=3

The number of replicas eventually propagates to the desired and current state of the deployment configured by the deployment configuration frontend.

Note

Pods can also be autoscaled using the oc autoscale command. See Pod Autoscaling for more details.

9.2.11. Adding Secrets to Deployment Configurations from the Web Console

Add a secret to your deployment configuration so that it can access a private repository.

  1. Create a new OpenShift Online project.
  2. Create a secret that contains credentials for accessing a private image repository.
  3. Create a deployment configuration.
  4. On the deployment configuration editor page or in the fromimage page of the web console, set the Pull Secret.
  5. Click the Save button.

9.3. Deployment Strategies

9.3.1. What Are Deployment Strategies?

A deployment strategy is a way to change or upgrade an application. The aim is to make the change without downtime in a way that the user barely notices the improvements.

The most common strategy is to use a blue-green deployment. The new version (the blue version) is brought up for testing and evaluation, while the users still use the stable version (the green version). When ready, the users are switched to the blue version. If a problem arises, you can switch back to the green version.

A common alternative strategy is to use A/B versions that are both active at the same time and some users use one version, and some users use the other version. This can be used for experimenting with user interface changes and other features to get user feedback. It can also be used to verify proper operation in a production context where problems impact a limited number of users.

A canary deployment tests the new version but when a problem is detected it quickly falls back to the previous version. This can be done with both of the above strategies.

The route based deployment strategies do not scale the number of pods in the services. To maintain desired performance characteristics the deployment configurations may need to be scaled.

There are things to consider when choosing a deployment strategy.

  • Long running connections need to be handled gracefully.
  • Database conversions can get tricky and will need to be done and rolled back along with the application.
  • If the application is a hybrid of microservices and traditional components downtime may be needed to complete the transition.
  • You need the infrastructure to do this.
  • If you have a non-isolated test environment, you can break both new and old versions.

Since the end user usually accesses the application through a route handled by a router, the deployment strategy can focus on deployment configuration features or routing features.

Strategies that focus on the deployment configuration impact all routes that use the application. Strategies that use router features target individual routes.

Many deployment strategies are supported through the deployment configuration and some additional strategies are supported through router features. The deployment configuration-based strategies are discussed in this section.

The Rolling strategy is the default strategy used if no strategy is specified on a deployment configuration.

A deployment strategy uses readiness checks to determine if a new pod is ready for use. If a readiness check fails, the deployment configuration will retry to run the pod until it times out. The default timeout is 10m, a value set in TimeoutSeconds in dc.spec.strategy.*params.

9.3.2. Rolling Strategy

A rolling deployment slowly replaces instances of the previous version of an application with instances of the new version of the application. A rolling deployment typically waits for new pods to become ready via a readiness check before scaling down the old components. If a significant issue occurs, the rolling deployment can be aborted.

9.3.2.1. Canary Deployments

All rolling deployments in OpenShift Online are canary deployments; a new version (the canary) is tested before all of the old instances are replaced. If the readiness check never succeeds, the canary instance is removed and the deployment configuration will be automatically rolled back. The readiness check is part of the application code, and may be as sophisticated as necessary to ensure the new instance is ready to be used. If you need to implement more complex checks of the application (such as sending real user workloads to the new instance), consider implementing a custom deployment or using a blue-green deployment strategy.

9.3.2.2. When to Use a Rolling Deployment

  • When you want to take no downtime during an application update.
  • When your application supports having old code and new code running at the same time.

A rolling deployment means you to have both old and new versions of your code running at the same time. This typically requires that your application handle N-1 compatibility.

The following is an example of the Rolling strategy:

strategy:
  type: Rolling
  rollingParams:
    timeoutSeconds: 120 1
    maxSurge: "20%" 2
    maxUnavailable: "10%" 3
    pre: {} 4
    post: {}
1
How long to wait for a scaling event before giving up. Optional; the default is 120.
2
maxSurge is optional and defaults to 25% if not specified; see below.
3
maxUnavailable is optional and defaults to 25% if not specified; see below.
4
pre and post are both lifecycle hooks.

The Rolling strategy will:

  1. Execute any pre lifecycle hook.
  2. Scale up the new replication controller based on the surge count.
  3. Scale down the old replication controller based on the max unavailable count.
  4. Repeat this scaling until the new replication controller has reached the desired replica count and the old replication controller has been scaled to zero.
  5. Execute any post lifecycle hook.
Important

When scaling down, the Rolling strategy waits for pods to become ready so it can decide whether further scaling would affect availability. If scaled up pods never become ready, the deployment process will eventually time out and result in a deployment failure.

The maxUnavailable parameter is the maximum number of pods that can be unavailable during the update. The maxSurge parameter is the maximum number of pods that can be scheduled above the original number of pods. Both parameters can be set to either a percentage (e.g., 10%) or an absolute value (e.g., 2). The default value for both is 25%.

These parameters allow the deployment to be tuned for availability and speed. For example:

  • maxUnavailable=0 and maxSurge=20% ensures full capacity is maintained during the update and rapid scale up.
  • maxUnavailable=10% and maxSurge=0 performs an update using no extra capacity (an in-place update).
  • maxUnavailable=10% and maxSurge=10% scales up and down quickly with some potential for capacity loss.

Generally, if you want fast rollouts, use maxSurge. If you need to take into account resource quota and can accept partial unavailability, use maxUnavailable.

9.3.2.3. Rolling Example

Rolling deployments are the default in OpenShift Online. To see a rolling update, follow these steps:

  1. Create an application based on the example deployment images found in DockerHub:

    $ oc new-app openshift/deployment-example

    If you have the router installed, make the application available via a route (or use the service IP directly)

    $ oc expose svc/deployment-example

    Browse to the application at deployment-example.<project>.<router_domain> to verify you see the v1 image.

  2. Scale the deployment configuration up to three replicas:

    $ oc scale dc/deployment-example --replicas=3
  3. Trigger a new deployment automatically by tagging a new version of the example as the latest tag:

    $ oc tag deployment-example:v2 deployment-example:latest
  4. In your browser, refresh the page until you see the v2 image.
  5. If you are using the CLI, the following command will show you how many pods are on version 1 and how many are on version 2. In the web console, you should see the pods slowly being added to v2 and removed from v1.

    $ oc describe dc deployment-example

During the deployment process, the new replication controller is incrementally scaled up. Once the new pods are marked as ready (by passing their readiness check), the deployment process will continue. If the pods do not become ready, the process will abort, and the deployment configuration will be rolled back to its previous version.

9.3.3. Recreate Strategy

The Recreate strategy has basic rollout behavior and supports lifecycle hooks for injecting code into the deployment process.

The following is an example of the Recreate strategy:

strategy:
  type: Recreate
  recreateParams: 1
    pre: {} 2
    mid: {}
    post: {}
1
recreateParams are optional.
2
pre, mid, and post are lifecycle hooks.

The Recreate strategy will:

  1. Execute any pre lifecycle hook.
  2. Scale down the previous deployment to zero.
  3. Execute any mid lifecycle hook.
  4. Scale up the new deployment.
  5. Execute any post lifecycle hook.
Important

During scale up, if the replica count of the deployment is greater than one, the first replica of the deployment will be validated for readiness before fully scaling up the deployment. If the validation of the first replica fails, the deployment will be considered a failure.

9.3.3.1. When to Use a Recreate Deployment

  • When you must run migrations or other data transformations before your new code starts.
  • When you do not support having new and old versions of your application code running at the same time.
  • When you want to use a RWO volume, which is not supported being shared between multiple replicas.

A recreate deployment incurs downtime because, for a brief period, no instances of your application are running. However, your old code and new code do not run at the same time.

9.3.4. Custom Strategy

The Custom strategy allows you to provide your own deployment behavior.

The following is an example of the Custom strategy:

strategy:
  type: Custom
  customParams:
    image: organization/strategy
    command: [ "command", "arg1" ]
    environment:
      - name: ENV_1
        value: VALUE_1

In the above example, the organization/strategy container image provides the deployment behavior. The optional command array overrides any CMD directive specified in the image’s Dockerfile. The optional environment variables provided are added to the execution environment of the strategy process.

Additionally, OpenShift Online provides the following environment variables to the deployment process:

Environment VariableDescription

OPENSHIFT_DEPLOYMENT_NAME

The name of the new deployment (a replication controller).

OPENSHIFT_DEPLOYMENT_NAMESPACE

The name space of the new deployment.

The replica count of the new deployment will initially be zero. The responsibility of the strategy is to make the new deployment active using the logic that best serves the needs of the user.

Learn more about advanced deployment strategies.

Alternatively, use customParams to inject the custom deployment logic into the existing deployment strategies. Provide a custom shell script logic and call the openshift-deploy binary. Users do not have to supply their custom deployer container image, but the default OpenShift Online deployer image will be used instead:

strategy:
  type: Rolling
  customParams:
    command:
    - /bin/sh
    - -c
    - |
      set -e
      openshift-deploy --until=50%
      echo Halfway there
      openshift-deploy
      echo Complete

This will result in following deployment:

Started deployment #2
--> Scaling up custom-deployment-2 from 0 to 2, scaling down custom-deployment-1 from 2 to 0 (keep 2 pods available, don't exceed 3 pods)
    Scaling custom-deployment-2 up to 1
--> Reached 50% (currently 50%)
Halfway there
--> Scaling up custom-deployment-2 from 1 to 2, scaling down custom-deployment-1 from 2 to 0 (keep 2 pods available, don't exceed 3 pods)
    Scaling custom-deployment-1 down to 1
    Scaling custom-deployment-2 up to 2
    Scaling custom-deployment-1 down to 0
--> Success
Complete

If the custom deployment strategy process requires access to the OpenShift Online API or the Kubernetes API the container that executes the strategy can use the service account token available inside the container for authentication.

9.3.5. Lifecycle Hooks

The Recreate and Rolling strategies support lifecycle hooks, which allow behavior to be injected into the deployment process at predefined points within the strategy:

The following is an example of a pre lifecycle hook:

pre:
  failurePolicy: Abort
  execNewPod: {} 1

Every hook has a failurePolicy, which defines the action the strategy should take when a hook failure is encountered:

Abort

The deployment process will be considered a failure if the hook fails.

Retry

The hook execution should be retried until it succeeds.

Ignore

Any hook failure should be ignored and the deployment should proceed.

Hooks have a type-specific field that describes how to execute the hook. Currently, pod-based hooks are the only supported hook type, specified by the execNewPod field.

9.3.5.1. Pod-based Lifecycle Hook

Pod-based lifecycle hooks execute hook code in a new pod derived from the template in a deployment configuration.

The following simplified example deployment configuration uses the Rolling strategy. Triggers and some other minor details are omitted for brevity:

kind: DeploymentConfig
apiVersion: v1
metadata:
  name: frontend
spec:
  template:
    metadata:
      labels:
        name: frontend
    spec:
      containers:
        - name: helloworld
          image: openshift/origin-ruby-sample
  replicas: 5
  selector:
    name: frontend
  strategy:
    type: Rolling
    rollingParams:
      pre:
        failurePolicy: Abort
        execNewPod:
          containerName: helloworld 1
          command: [ "/usr/bin/command", "arg1", "arg2" ] 2
          env: 3
            - name: CUSTOM_VAR1
              value: custom_value1
          volumes:
            - data 4
1
The helloworld name refers to spec.template.spec.containers[0].name.
2
This command overrides any ENTRYPOINT defined by the openshift/origin-ruby-sample image.
3
env is an optional set of environment variables for the hook container.
4
volumes is an optional set of volume references for the hook container.

In this example, the pre hook will be executed in a new pod using the openshift/origin-ruby-sample image from the helloworld container. The hook pod will have the following properties:

  • The hook command will be /usr/bin/command arg1 arg2.
  • The hook container will have the CUSTOM_VAR1=custom_value1 environment variable.
  • The hook failure policy is Abort, meaning the deployment process will fail if the hook fails.
  • The hook pod will inherit the data volume from the deployment configuration pod.

9.3.5.2. Using the Command Line

The oc set deployment-hook command can be used to set the deployment hook for a deployment configuration. For the example above, you can set the pre-deployment hook with the following command:

$ oc set deployment-hook dc/frontend --pre -c helloworld -e CUSTOM_VAR1=custom_value1 \
  -v data --failure-policy=abort -- /usr/bin/command arg1 arg2

9.4. Advanced Deployment Strategies

9.4.1. Advanced Deployment Strategies

Deployment strategies provide a way for the application to evolve. Some strategies use the deployment configuration to make changes that are seen by users of all routes that resolve to the application. Other strategies, such as the ones described here, use router features to impact specific routes.

9.4.2. Blue-Green Deployment

Blue-green deployments involve running two versions of an application at the same time and moving traffic from the in-production version (the green version) to the newer version (the blue version). You can use a rolling strategy or switch services in a route.

Note

Since many applications depend on persistent data, you will need to have an application that supports N-1 compatibility, which means you share data and implement live migration between your database, store, or disk by creating two copies of your data layer.

Consider the data used in testing the new version. If it is the production data, a bug in the new version can break the production version.

9.4.2.1. Using a Blue-Green Deployment

Blue-Green deployments use two deployment configurations. Both are running, and the one in production depends on the service the route specifies, with each deployment configuration exposed to a different service. You can create a new route to the new version and test it. When ready, change the service in the production route to point to the new service and the new, blue, version is live.

If necessary, you can roll back to the older, green, version by switching service back to the previous version.

Using a Route and Two Services

This example sets up two deployment configurations; one for the stable version (the green version) and the other for the newer version (the blue version).

A route points to a service, and can be changed to point to a different service at any time. As a developer, you can test the new version of your code by connecting to the new service before your production traffic is routed to it.

Routes are intended for web (HTTP and HTTPS) traffic, so this technique is best suited for web applications.

  1. Create two copies of the example application:

    $ oc new-app openshift/deployment-example:v1 --name=example-green
    $ oc new-app openshift/deployment-example:v2 --name=example-blue

    This creates two independent application components: one running the v1 image under the example-green service, and one using the v2 image under the example-blue service.

  2. Create a route that points to the old service:

    $ oc expose svc/example-green --name=bluegreen-example
  3. Browse to the application at bluegreen-example.<project>.<router_domain> to verify you see the v1 image.
  4. Edit the route and change the service name to example-blue:

    $ oc patch route/bluegreen-example -p '{"spec":{"to":{"name":"example-blue"}}}'
  5. To verify that the route has changed, refresh the browser until you see the v2 image.

9.4.3. A/B Deployment

The A/B deployment strategy lets you try a new version of the application in a limited way in the production environment. You can specify that the production version gets most of the user requests while a limited fraction of requests go to the new version. Since you control the portion of requests to each version, as testing progresses you can increase the fraction of requests to the new version and ultimately stop using the previous version. As you adjust the request load on each version, the number of pods in each service may need to be scaled as well to provide the expected performance.

In addition to upgrading software, you can use this feature to experiment with versions of the user interface. Since some users get the old version and some the new, you can evaluate the user’s reaction to the different versions to inform design decisions.

For this to be effective, both the old and new versions need to be similar enough that both can run at the same time. This is common with bug fix releases and when new features do not interfere with the old. The versions need N-1 compatibility to properly work together.

OpenShift Online supports N-1 compatibility through the web console as well as the command line interface.

9.4.3.1. Load Balancing for A/B Testing

The user sets up a route with multiple services. Each service handles a version of the application.

Each service is assigned a weight and the portion of requests to each service is the service_weight divided by the sum_of_weights. The weight for each service is distributed to the service’s endpoints so that the sum of the endpoint weights is the service weight.

The route can have up to four services. The weight for the service can be between 0 and 256. When the weight is 0, no new requests go to the service, however existing connections remain active. When the service weight is not 0, each endpoint has a minimum weight of 1. Because of this, a service with a lot of endpoints can end up with higher weight than desired. In this case, reduce the number of pods to get the desired load balance weight. See the Alternate Backends and Weights section for more information.

The web console allows users to set the weighting and show balance between them:

Visualization of Alternate Back Ends in the Web Console

To set up the A/B environment:

  1. Create the two applications and give them different names. Each creates a deployment configuration. The applications are versions of the same program; one is usually the current production version and the other the proposed new version:

    $ oc new-app openshift/deployment-example1 --name=ab-example-a
    $ oc new-app openshift/deployment-example2 --name=ab-example-b
  2. Expose the deployment configuration to create a service:

    $ oc expose dc/ab-example-a --name=ab-example-A
    $ oc expose dc/ab-example-b --name=ab-example-B

    At this point both applications are deployed and are running and have services.

  3. Make the application available externally via a route. You can expose either service at this point, it may be convenient to expose the current production version and latter modify the route to add the new version.

    $ oc expose svc/ab-example-A

    Browse to the application at ab-example.<project>.<router_domain> to verify that you see the desired version.

  4. When you deploy the route, the router will balance the traffic according to the weights specified for the services. At this point there is a single service with default weight=1 so all requests go to it. Adding the other service as an alternateBackends and adjusting the weights will bring the A/B setup to life. This can be done by the oc set route-backends command or by editing the route.

    Note

    Changes to the route just change the portion of traffic to the various services. You may need to scale the deployment configurations to adjust the number of pods to handle the anticipated loads.

    To edit the route, run:

    $ oc edit route <route-name>
    ...
    metadata:
      name: route-alternate-service
      annotations:
        haproxy.router.openshift.io/balance: roundrobin
    spec:
      host: ab-example.my-project.my-domain
      to:
        kind: Service
        name: ab-example-A
        weight: 10
      alternateBackends:
      - kind: Service
        name: ab-example-B
        weight: 15
    ...
9.4.3.1.1. Managing Weights

This command manages the services and corresponding weightsload balanced by the route.

  oc set route-backends ROUTENAME [--zero|--equal] [--adjust] SERVICE=WEIGHT[%] [...] [options]

For example, the following sets ab-example-A as the primary service with weight=198 and ab-example-B as the first alternate service with a weight=2:

$ oc set route-backends web ab-example-A=198 ab-example-B=2

This means 99% of traffic will be sent to service ab-example-A and 1% to service ab-example-B.

This command does not scale the deployment configurations. You may need to do that to have enough pods to handle the request load.

The command with no flags displays the current configuration.

$ oc set route-backends web
NAME                    KIND     TO           WEIGHT
routes/web              Service  ab-example-A 198 (99%)
routes/web              Service  ab-example-B 2   (1%)

The --adjust flag allows you to alter the weight of an individual service relative to itself or to the primary service. Specifying a percentage will adjust the service relative to either the primary or the first alternate (if you specify the primary). If there are other backends their weights will be kept proportional to the changed.

$ oc rollout latest dc/ab-example-a

The --equal flag sets the weight of all services to 100

$ oc set route-backends web --equal

The --zero flag sets the weight of all services to 0. All requests will return with a 503 error.

Note

Not all routers may support multiple or weighted backends.

9.4.3.1.2. One Service, Multiple Deployment Configurations

If you have the router installed, make the application available via a route (or use the service IP directly):

$ oc expose svc/ab-example

Browse to the application at ab-example.<project>.<router_domain> to verify you see the v1 image.

  1. Create a second shard based on the same source image as the first shard but different tagged version, and set a unique value:

    $ oc new-app openshift/deployment-example:v2 --name=ab-example-b --labels=ab-example=true SUBTITLE="shard B" COLOR="red"
  2. Edit the newly created shard to set a label ab-example=true that will be common to all shards:

    $ oc edit dc/ab-example-b

    In the editor, add the line ab-example: "true" underneath spec.selector and spec.template.metadata.labels alongside the existing deploymentconfig=ab-example-b label. Save and exit the editor.

  3. Trigger a re-deployment of the second shard to pick up the new labels:

    $ oc rollout latest dc/ab-example-b
  4. At this point, both sets of pods are being served under the route. However, since both browsers (by leaving a connection open) and the router (by default, through a cookie) will attempt to preserve your connection to a back-end server, you may not see both shards being returned to you. To force your browser to one or the other shard, use the scale command:

    $ oc scale dc/ab-example-a --replicas=0

    Refreshing your browser should show v2 and shard B (in red).

    $ oc scale dc/ab-example-a --replicas=1; oc scale dc/ab-example-b --replicas=0

    Refreshing your browser should show v1 and shard A (in blue).

    If you trigger a deployment on either shard, only the pods in that shard will be affected. You can easily trigger a deployment by changing the SUBTITLE environment variable in either deployment config oc edit dc/ab-example-a or oc edit dc/ab-example-b. You can add additional shards by repeating steps 5-7.

    Note

    These steps will be simplified in future versions of OpenShift Online.

9.4.4. Proxy Shard / Traffic Splitter

In production environments, you can precisely control the distribution of traffic that lands on a particular shard. When dealing with large numbers of instances, you can use the relative scale of individual shards to implement percentage based traffic. That combines well with a proxy shard, which forwards or splits the traffic it receives to a separate service or application running elsewhere.

In the simplest configuration, the proxy would forward requests unchanged. In more complex setups, you can duplicate the incoming requests and send to both a separate cluster as well as to a local instance of the application, and compare the result. Other patterns include keeping the caches of a DR installation warm, or sampling incoming traffic for analysis purposes.

While an implementation is beyond the scope of this example, any TCP (or UDP) proxy could be run under the desired shard. Use the oc scale command to alter the relative number of instances serving requests under the proxy shard. For more complex traffic management, consider customizing the OpenShift Online router with proportional balancing capabilities.

9.4.5. N-1 Compatibility

Applications that have new code and old code running at the same time must be careful to ensure that data written by the new code can be read and handled (or gracefully ignored) by the old version of the code. This is sometimes called schema evolution and is a complex problem.

This can take many forms — data stored on disk, in a database, in a temporary cache, or that is part of a user’s browser session. While most web applications can support rolling deployments, it is important to test and design your application to handle it.

For some applications, the period of time that old code and new code is running side by side is short, so bugs or some failed user transactions are acceptable. For others, the failure pattern may result in the entire application becoming non-functional.

One way to validate N-1 compatibility is to use an A/B deployment. Run the old code and new code at the same time in a controlled way in a test environment, and verify that traffic that flows to the new deployment does not cause failures in the old deployment.

9.4.6. Graceful Termination

OpenShift Online and Kubernetes give application instances time to shut down before removing them from load balancing rotations. However, applications must ensure they cleanly terminate user connections as well before they exit.

On shutdown, OpenShift Online will send a TERM signal to the processes in the container. Application code, on receiving SIGTERM, should stop accepting new connections. This will ensure that load balancers route traffic to other active instances. The application code should then wait until all open connections are closed (or gracefully terminate individual connections at the next opportunity) before exiting.

After the graceful termination period expires, a process that has not exited will be sent the KILL signal, which immediately ends the process. The terminationGracePeriodSeconds attribute of a pod or pod template controls the graceful termination period (default 30 seconds) and may be customized per application as necessary.

Chapter 10. Templates

10.1. Overview

A template describes a set of objects that can be parameterized and processed to produce a list of objects for creation by OpenShift Online. A template can be processed to create anything you have permission to create within a project, for example services, build configurations, and deployment configurations. A template may also define a set of labels to apply to every object defined in the template.

You can create a list of objects from a template using the CLI or, if a template has been uploaded to your project or the global template library, using the web console. For a curated set of templates, see the OpenShift Image Streams and Templates library.

10.2. Uploading a Template

If you have a JSON or YAML file that defines a template, for example as seen in this example, you can upload the template to projects using the CLI. This saves the template to the project for repeated use by any user with appropriate access to that project. Instructions on writing your own templates are provided later in this topic.

To upload a template to your current project’s template library, pass the JSON or YAML file with the following command:

$ oc create -f <filename>

You can upload a template to a different project using the -n option with the name of the project:

$ oc create -f <filename> -n <project>

The template is now available for selection using the web console or the CLI.

10.3. Creating from Templates Using the Web Console

To create the objects from an uploaded template using the web console:

  1. While in the desired project, click Add to Project:

    Console Create
  2. Select a template from the list of templates in your project, or provided by the global template library:

    Select Template
  3. Modify template parameters in the template creation screen:

    Create from Template
    Template name and description.
    Container images included in the template.
    Parameters defined by the template. You can edit values for parameters defined in the template here.
    Labels to assign to all items included in the template. You can add and edit labels for objects.

10.4. Creating from Templates Using the CLI

You can use the CLI to process templates and use the configuration that is generated to create objects.

10.4.1. Labels

Labels are used to manage and organize generated objects, such as pods. The labels specified in the template are applied to every object that is generated from the template.

There is also the ability to add labels in the template from the command line.

$ oc process -f <filename> -l name=otherLabel

10.4.2. Parameters

The list of parameters that you can override are listed in the parameters section of the template. You can list them with the CLI by using the following command and specifying the file to be used:

$ oc process --parameters -f <filename>

Alternatively, if the template is already uploaded:

$ oc process --parameters -n <project> <template_name>

For example, the following shows the output when listing the parameters for one of the Quickstart templates in the default openshift project:

$ oc process --parameters -n openshift rails-postgresql-example
NAME                         DESCRIPTION                                                                                              GENERATOR           VALUE
SOURCE_REPOSITORY_URL        The URL of the repository with your application source code                                                                  https://github.com/openshift/rails-ex.git
SOURCE_REPOSITORY_REF        Set this to a branch name, tag or other ref of your repository if you are not using the default branch
CONTEXT_DIR                  Set this to the relative path to your project if it is not in the root of your repository
APPLICATION_DOMAIN           The exposed hostname that will route to the Rails service                                                                    rails-postgresql-example.openshiftapps.com
GITHUB_WEBHOOK_SECRET        A secret string used to configure the GitHub webhook                                                     expression          [a-zA-Z0-9]{40}
SECRET_KEY_BASE              Your secret key for verifying the integrity of signed cookies                                            expression          [a-z0-9]{127}
APPLICATION_USER             The application user that is used within the sample application to authorize access on pages                                 openshift
APPLICATION_PASSWORD         The application password that is used within the sample application to authorize access on pages                             secret
DATABASE_SERVICE_NAME        Database service name                                                                                                        postgresql
POSTGRESQL_USER              database username                                                                                        expression          user[A-Z0-9]{3}
POSTGRESQL_PASSWORD          database password                                                                                        expression          [a-zA-Z0-9]{8}
POSTGRESQL_DATABASE          database name                                                                                                                root
POSTGRESQL_MAX_CONNECTIONS   database max connections                                                                                                     10
POSTGRESQL_SHARED_BUFFERS    database shared buffers                                                                                                      12MB

The output identifies several parameters that are generated with a regular expression-like generator when the template is processed.

10.4.3. Generating a List of Objects

Using the CLI, you can process a file defining a template to return the list of objects to standard output:

$ oc process -f <filename>

Alternatively, if the template has already been uploaded to the current project:

$ oc process <template_name>

You can create objects from a template by processing the template and piping the output to oc create:

$ oc process -f <filename> | oc create -f -

Alternatively, if the template has already been uploaded to the current project:

$ oc process <template> | oc create -f -

You can override any parameter values defined in the file by adding the -p option for each <name>=<value> pair you want to override. A parameter reference may appear in any text field inside the template items.

For example, in the following the POSTGRESQL_USER and POSTGRESQL_DATABASE parameters of a template are overridden to output a configuration with customized environment variables:

Example 10.1. Creating a List of Objects from a Template

$ oc process -f my-rails-postgresql \
    -p POSTGRESQL_USER=bob \
    -p POSTGRESQL_DATABASE=mydatabase

The JSON file can either be redirected to a file or applied directly without uploading the template by piping the processed output to the oc create command:

$ oc process -f my-rails-postgresql \
    -p POSTGRESQL_USER=bob \
    -p POSTGRESQL_DATABASE=mydatabase \
    | oc create -f -

If you have large number of parameters, you can store them in a file and then pass this file to oc process:

$ cat postgres.env
POSTGRESQL_USER=bob
POSTGRESQL_DATABASE=mydatabase
$ oc process -f my-rails-postgresql --param-file=postgres.env

You can also read the environment from standard input by using "-" as the argument to --param-file:

$ sed s/bob/alice/ postgres.env | oc process -f my-rails-postgresql --param-file=-

10.5. Modifying an Uploaded Template

You can edit a template that has already been uploaded to your project by using the following command:

$ oc edit template <template>

10.6. Using the Instant App and Quickstart Templates

OpenShift Online provides a number of default Instant App and Quickstart templates to make it easy to quickly get started creating a new application for different languages. Templates are provided for Rails (Ruby), Django (Python), Node.js, CakePHP (PHP), and Dancer (Perl). Your cluster administrator should have created these templates in the default, global openshift project so you have access to them. You can list the available default Instant App and Quickstart templates with:

$ oc get templates -n openshift

By default, the templates build using a public source repository on GitHub that contains the necessary application code. In order to be able to modify the source and build your own version of the application, you must:

  1. Fork the repository referenced by the template’s default SOURCE_REPOSITORY_URL parameter.
  2. Override the value of the SOURCE_REPOSITORY_URL parameter when creating from the template, specifying your fork instead of the default value.

By doing this, the build configuration created by the template will now point to your fork of the application code, and you can modify the code and rebuild the application at will.

Note

Some of the Instant App and Quickstart templates define a database deployment configuration. The configuration they define uses ephemeral storage for the database content. These templates should be used for demonstration purposes only as all database data will be lost if the database pod restarts for any reason.

10.7. Writing Templates

You can define new templates to make it easy to recreate all the objects of your application. The template will define the objects it creates along with some metadata to guide the creation of those objects.

Example 10.2. A Simple Template Object Definition (YAML)

apiVersion: v1
kind: Template
metadata:
  name: redis-template
  annotations:
    description: "Description"
    iconClass: "icon-redis"
    tags: "database,nosql"
objects:
- apiVersion: v1
  kind: Pod
  metadata:
    name: redis-master
  spec:
    containers:
    - env:
      - name: REDIS_PASSWORD
        value: ${REDIS_PASSWORD}
      image: dockerfile/redis
      name: master
      ports:
      - containerPort: 6379
        protocol: TCP
parameters:
- description: Password used for Redis authentication
  from: '[A-Z0-9]{8}'
  generate: expression
  name: REDIS_PASSWORD
labels:
  redis: master

10.7.1. Description

The template description informs users what the template does and helps them find it when searching in the web console. Additional metadata beyond the template name is optional, but useful to have. In addition to general descriptive information, the metadata also includes a set of tags. Useful tags include the name of the language the template is related to (for example, java, php, ruby, and so on).

Example 10.3. Template Description Metadata

kind: Template
apiVersion: v1
metadata:
  name: cakephp-mysql-example 1
  annotations:
    openshift.io/display-name: "CakePHP MySQL Example (Ephemeral)" 2
    description: >-
      An example CakePHP application with a MySQL database. For more information
      about using this template, including OpenShift considerations, see
      https://github.com/openshift/cakephp-ex/blob/master/README.md.


      WARNING: Any data stored will be lost upon pod destruction. Only use this
      template for testing." 3
    template.openshift.io/long-description: >-
      This template defines resources needed to develop a CakePHP application,
      including a build configuration, application deployment configuration, and
      database deployment configuration.  The database is stored in
      non-persistent storage, so this configuration should be used for
      experimental purposes only. 4
    tags: "quickstart,php,cakephp" 5
    iconClass: icon-php 6
    template.openshift.io/provider-display-name: "Red Hat, Inc." 7
    template.openshift.io/documentation-url: "https://github.com/openshift/cakephp-ex" 8
    template.openshift.io/support-url: "https://access.redhat.com" 9
message: "Your admin credentials are ${ADMIN_USERNAME}:${ADMIN_PASSWORD}" 10
1
The unique name of the template.
2
A brief, user-friendly name, which can be employed by user interfaces.
3
A description of the template. Include enough detail that the user will understand what is being deployed and any caveats they need to know before deploying. It should also provide links to additional information, such as a README file. Newlines can be included to create paragraphs.
4
Additional template description. This may be displayed by the service catalog, for example.
5
Tags to be associated with the template for searching and grouping. Add tags that will include it into one of the provided catalog categories. Refer to the id and categoryAliases in CATALOG_CATEGORIES in the console’s constants file.
6
An icon to be displayed with your template in the web console. Choose from our existing logo icons when possible. You can also use icons from FontAwesome and Patternfly.
7
The name of the person or organization providing the template.
8
A URL referencing further documentation for the template.
9
A URL where support can be obtained for the template.
10
An instructional message that is displayed when this template is instantiated. This field should inform the user how to use the newly created resources. Parameter substitution is performed on the message before being displayed so that generated credentials and other parameters can be included in the output. Include links to any next-steps documentation that users should follow.

10.7.2. Labels

Templates can include a set of labels. These labels will be added to each object created when the template is instantiated. Defining a label in this way makes it easy for users to find and manage all the objects created from a particular template.

Example 10.4. Template Object Labels

kind: "Template"
apiVersion: "v1"
...
labels:
  template: "cakephp-mysql-example" 1
1
A label that will be applied to all objects created from this template.

10.7.3. Parameters

Parameters allow a value to be supplied by the user or generated when the template is instantiated. Then, that value is substituted wherever the parameter is referenced. References can be defined in any field in the objects list field. This is useful for generating random passwords or allowing the user to supply a host name or other user-specific value that is required to customize the template. Parameters can be referenced in two ways:

  • As a string value by placing values in the form ${PARAMETER_NAME} in any string field in the template.
  • As a json/yaml value by placing values in the form ${{PARAMETER_NAME}} in place of any field in the template.

When using the ${PARAMETER_NAME} syntax, multiple parameter references can be combined in a single field and the reference can be embedded within fixed data, such as "http://{PARAMETER_1}#{$PARAMETER_2}". Both parameter values will be substituted and the resulting value will be a quoted string.

When using the ${{PARAMETER_NAME}} syntax only a single parameter reference is allowed and leading/trailing characters are not permitted. The resulting value will be unquoted unless, after substitution is performed, the result is not a valid json object. If the result is not a valid json value, the resulting value will be quoted and treated as a standard string.

A single parameter can be referenced multiple times within a template and it can be referenced using both substitution syntaxes within a single template.

A default value can be provided, which is used if the user does not supply a different value:

Example 10.5. Setting an Explicit Value as the Default Value

parameters:
  - name: USERNAME
    description: "The user name for Joe"
    value: joe

Parameter values can also be generated based on rules specified in the parameter definition:

Example 10.6. Generating a Parameter Value

parameters:
  - name: PASSWORD
    description: "The random user password"
    generate: expression
    from: "[a-zA-Z0-9]{12}"

In the example above, processing will generate a random password 12 characters long consisting of all upper and lowercase alphabet letters and numbers.

The syntax available is not a full regular expression syntax. However, you can use \w, \d, and \a modifiers:

  • [\w]{10} produces 10 alphabet characters, numbers, and underscores. This follows the PCRE standard and is equal to [a-zA-Z0-9_]{10}.
  • [\d]{10} produces 10 numbers. This is equal to [0-9]{10}.
  • [\a]{10} produces 10 alphabetical characters. This is equal to [a-zA-Z]{10}.

Here is an example of a full template with parameter definitions and references:

Example 10.7. A full template with parameter definitions and references

kind: Template
apiVersion: v1
objects:
  - kind: BuildConfig
    apiVersion: v1
    metadata:
      name: cakephp-mysql-example
      annotations:
        description: Defines how to build the application
    spec:
      source:
        type: Git
        git:
          uri: "${SOURCE_REPOSITORY_URL}" 1
          ref: "${SOURCE_REPOSITORY_REF}"
        contextDir: "${CONTEXT_DIR}"
  - kind: DeploymentConfig
    apiVersion: v1
    metadata:
      name: frontend
    spec:
      replicas: "${{REPLICA_COUNT}}" 2
parameters:
  - name: SOURCE_REPOSITORY_URL 3
    displayName: Source Repository URL 4
    description: The URL of the repository with your application source code 5
    value: https://github.com/openshift/cakephp-ex.git 6
    required: true 7
  - name: GITHUB_WEBHOOK_SECRET
    description: A secret string used to configure the GitHub webhook
    generate: expression 8
    from: "[a-zA-Z0-9]{40}" 9
  - name: REPLICA_COUNT
    description: Number of replicas to run
    value: "2"
    required: true
message: "... The GitHub webhook secret is ${GITHUB_WEBHOOK_SECRET} ..." 10
1
This value will be replaced with the value of the SOURCE_REPOSITORY_URL parameter when the template is instantiated.
2
This value will be replaced with the unquoted value of the REPLICA_COUNT parameter when the template is instantiated.
3
The name of the parameter. This value is used to reference the parameter within the template.
4
The user-friendly name for the parameter. This will be displayed to users.
5
A description of the parameter. Provide more detailed information for the purpose of the parameter, including any constraints on the expected value. Descriptions should use complete sentences to follow the console’s text standards. Don’t make this a duplicate of the display name.
6
A default value for the parameter which will be used if the user does not override the value when instantiating the template. Avoid using default values for things like passwords, instead use generated parameters in combination with Secrets.
7
Indicates this parameter is required, meaning the user cannot override it with an empty value. If the parameter does not provide a default or generated value, the user must supply a value.
8
A parameter which has its value generated.
9
The input to the generator. In this case, the generator will produce a 40 character alphanumeric value including upper and lowercase characters.
10
Parameters can be included in the template message. This informs the user about generated values.

10.7.4. Object List

The main portion of the template is the list of objects which will be created when the template is instantiated. This can be any valid API object, such as a BuildConfig, DeploymentConfig, Service, etc. The object will be created exactly as defined here, with any parameter values substituted in prior to creation. The definition of these objects can reference parameters defined earlier.

kind: "Template"
apiVersion: "v1"
objects:
  - kind: "Service" 1
    apiVersion: "v1"
    metadata:
      name: "cakephp-mysql-example"
      annotations:
        description: "Exposes and load balances the application pods"
    spec:
      ports:
        - name: "web"
          port: 8080
          targetPort: 8080
      selector:
        name: "cakephp-mysql-example"
1
The definition of a Service which will be created by this template.
Note

If an object definition’s metadata includes a fixed namespace field value, the field will be stripped out of the definition during template instantiation. If the namespace field contains a parameter reference, normal parameter substitution will be performed and the object will be created in whatever namespace the parameter substitution resolved the value to, assuming the user has permission to create objects in that namespace.

10.7.5. Exposing Object Fields

Template authors can indicate that fields of particular objects in a template should be exposed. The template service broker recognizes exposed fields on ConfigMap, Secret, Service and Route objects, and returns the values of the exposed fields when a user binds a service backed by the broker.

To expose one or more fields of an object, add annotations prefixed by template.openshift.io/expose- or template.openshift.io/base64-expose- to the object in the template.

Each annotation key, with its prefix removed, is passed through to become a key in a bind response.

Each annotation value is a Kubernetes JSONPath expression, which is resolved at bind time to indicate the object field whose value should be returned in the bind response.

Note

Bind response key/value pairs can be used in other parts of the system as environment variables. Therefore, it is recommended that every annotation key with its prefix removed should be a valid environment variable name — beginning with a character A-Z, a-z, or _, and being followed by zero or more characters A-Z, a-z, 0-9, or _.

Use the template.openshift.io/expose- annotation to return the field value as a string. This is convenient, although it does not handle arbitrary binary data. If you want to return binary data, use the template.openshift.io/base64-expose- annotation instead to base64 encode the data before it is returned.

Note

Unless escaped with a backslash, Kubernetes' JSONPath implementation interprets characters such as ., @, and others as metacharacters, regardless of their position in the expression. Therefore, for example, to refer to a ConfigMap datum named my.key, the required JSONPath expression would be {.data['my\.key']}.

The following is an example of different objects' fields being exposed:

kind: Template
apiVersion: v1
objects:
- kind: ConfigMap
  apiVersion: v1
  metadata:
    annotations:
      template.openshift.io/expose-username: "{.data['my\.username']}"
  data:
    my.username: foo
- kind: Secret
  apiVersion: v1
  metadata:
    annotations:
      template.openshift.io/base64-expose-password: "{.data['password']}"
  stringData:
    password: bar
- kind: Service
  apiVersion: v1
  metadata:
    annotations:
      template.openshift.io/expose-service_ip_port: "{.spec.clusterIP}:{.spec.ports[?(.name==\"web\")].port}"
  spec:
    ports:
    - name: "web"
      port: 8080
- kind: Route
  apiVersion: v1
  metadata:
    annotations:
      template.openshift.io/expose-uri: "http://{.spec.host}{.spec.path}"
  spec:
    path: mypath

An example response to a bind operation given the above partial template follows:

{
  "credentials": {
    "username": "foo",
    "password": "YmFy",
    "service_ip_port": "172.30.12.34:8080",
    "uri": "http://route-test.router.default.svc.cluster.local/mypath"
  }
}

10.7.6. Other Recommendations

  • Group related services together in the management console by adding the service.alpha.openshift.io/dependencies annotation to the Service object in your template.

    Example 10.8. Group the Frontend and Database Services Together on the Management Console Overview

    kind: "Template"
    apiVersion: "v1"
    objects:
      - kind: "Service"
        apiVersion: "v1"
        metadata:
          name: "frontend"
          annotations:
            "service.alpha.openshift.io/dependencies": "[{\"name\": \"database\", \"kind\": \"Service\"}]"
    ...
      - kind: "Service"
        apiVersion: "v1"
        metadata:
          name: "database"
  • Set memory, CPU, and storage default sizes to make sure your application is given enough resources to run smoothly.
  • Avoid referencing the latest tag from images if that tag is used across major versions. This may cause running applications to break when new images are pushed to that tag.
  • A good template builds and deploys cleanly without requiring modifications after the template is deployed.

10.7.7. Creating a Template from Existing Objects

If you are upgrading from OpenShift Online Starter to OpenShift Online Pro, use oc export all to export all of your existing objects. OpenShift Online Pro does not support per-object resource migration.

Rather than writing an entire template from scratch, you can export existing objects from your project in template form, and then modify the template from there by adding parameters and other customizations. To export objects in a project in template form, run:

$ oc export all --as-template=<template_name> > <template_filename>

You can also substitute a particular resource type or multiple resources instead of all. Run oc export -h for more examples.

The object types included in oc export all are:

  • BuildConfig
  • Build
  • DeploymentConfig
  • ImageStream
  • Pod
  • ReplicationController
  • Route
  • Service

Chapter 11. Opening a Remote Shell to Containers

11.1. Overview

The oc rsh command allows you to locally access and manage tools that are on the system. The secure shell (SSH) is the underlying technology and industry standard that provides a secure connection to the application. Access to applications with the shell environment is protected and restricted with Security-Enhanced Linux (SELinux) policies.

11.2. Start a Secure Shell Session

Open a remote shell session to a container:

$ oc rsh <pod>

While in the remote shell, you can issue commands as if you are inside the container and perform local operations like monitoring, debugging, and using CLI commands specific to what is running in the container.

For example, in a MySQL container, you can count the number of records in the database by invoking the mysql command, then using the the prompt to type in the SELECT command. You can also use use commands like ps(1) and ls(1) for validation.

BuildConfigs and DeployConfigs map out how you want things to look and pods (with containers inside) are created and dismantled as needed. Your changes are not persistent. If you make changes directly within the container and that container is destroyed and rebuilt, your changes will no longer exist.

Note

oc exec can be used to execute a command remotely. However, the oc rsh command provides an easier way to keep a remote shell open persistently.

11.3. Secure Shell Session Help

For help with usage, options, and to see examples:

$ oc rsh -h

Chapter 12. Service Accounts

12.1. Overview

When a person uses the OpenShift Online CLI or web console, their API token authenticates them to the OpenShift API. However, when a regular user’s credentials are not available, it is common for components to make API calls independently. For example:

  • Replication controllers make API calls to create or delete pods.
  • Applications inside containers could make API calls for discovery purposes.
  • External applications could make API calls for monitoring or integration purposes.

Service accounts provide a flexible way to control API access without sharing a regular user’s credentials.

12.2. User Names and Groups

Every service account has an associated user name that can be granted roles, just like a regular user. The user name is derived from its project and name:

system:serviceaccount:<project>:<name>

For example, to add the view role to the robot service account in the top-secret project:

$ oc policy add-role-to-user view system:serviceaccount:top-secret:robot

Every service account is also a member of two groups:

system:serviceaccounts
Includes all service accounts in the system.
system:serviceaccounts:<project>
Includes all service accounts in the specified project.

For example, to allow all service accounts in all projects to view resources in the top-secret project:

$ oc policy add-role-to-group view system:serviceaccounts -n top-secret

To allow all service accounts in the managers project to edit resources in the top-secret project:

$ oc policy add-role-to-group edit system:serviceaccounts:managers -n top-secret

12.3. Default Service Accounts and Roles

Three service accounts are automatically created in every project:

Service AccountUsage

builder

Used by build pods. It is given the system:image-builder role, which allows pushing images to any image stream in the project using the internal Docker registry.

deployer

Used by deployment pods and is given the system:deployer role, which allows viewing and modifying replication controllers and pods in the project.

default

Used to run all other pods unless they specify a different service account.

All service accounts in a project are given the system:image-puller role, which allows pulling images from any image stream in the project using the internal Docker registry.

12.4. Managing Service Accounts

Service accounts are API objects that exist within each project. To manage service accounts, you can use the oc command with the sa or serviceaccount object type or use the web console.

To get a list of existing service accounts in the current project:

$ oc get sa
NAME       SECRETS   AGE
builder    2         2d
default    2         2d
deployer   2         2d

To create a new service account:

$ oc create sa robot
serviceaccount "robot" created

As soon as a service account is created, two secrets are automatically added to it:

  • an API token
  • credentials for the OpenShift Container Registry

These can be seen by describing the service account:

$ oc describe sa robot
Name:		robot
Namespace:	project1
Labels:		<none>
Annotations:	<none>

Image pull secrets:	robot-dockercfg-qzbhb

Mountable secrets: 	robot-token-f4khf
                   	robot-dockercfg-qzbhb

Tokens:            	robot-token-f4khf
                   	robot-token-z8h44

The system ensures that service accounts always have an API token and registry credentials.

The generated API token and registry credentials do not expire, but they can be revoked by deleting the secret. When the secret is deleted, a new one is automatically generated to take its place.

12.5. Enabling Service Account Authentication

Service accounts authenticate to the API using tokens signed by a private RSA key. The authentication layer verifies the signature using a matching public RSA key.

To enable service account token generation, update the serviceAccountConfig stanza in the /etc/origin/master/master-config.yml file on the master to specify a privateKeyFile (for signing), and a matching public key file in the publicKeyFiles list:

serviceAccountConfig:
  ...
  masterCA: ca.crt 1
  privateKeyFile: serviceaccounts.private.key 2
  publicKeyFiles:
  - serviceaccounts.public.key 3
  - ...
1
CA file used to validate the API server’s serving certificate.
2
Private RSA key file (for token signing).
3
Public RSA key files (for token verification). If private key files are provided, then the public key component is used. Multiple public key files can be specified, and a token will be accepted if it can be validated by one of the public keys. This allows rotation of the signing key, while still accepting tokens generated by the previous signer.

12.6. Managing Allowed Secrets

In addition to providing API credentials, a pod’s service account determines which secrets the pod is allowed to use.

Pods use secrets in two ways:

  • image pull secrets, providing credentials used to pull images for the pod’s containers
  • mountable secrets, injecting the contents of secrets into containers as files

To allow a secret to be used as an image pull secret by a service account’s pods, run:

$ oc secrets link --for=pull <serviceaccount-name> <secret-name>

To allow a secret to be mounted by a service account’s pods, run:

$ oc secrets link --for=mount <serviceaccount-name> <secret-name>
Note

Limiting secrets to only the service accounts that reference them is disabled by default. This means that if serviceAccountConfig.limitSecretReferences is set to false (the default setting) in the master configuration file, mounting secrets to a service account’s pods with the --for=mount option is not required. However, using the --for=pull option to enable using an image pull secret is required, regardless of the serviceAccountConfig.limitSecretReferences value.

This example creates and adds secrets to a service account:

$ oc secrets new secret-plans plan1.txt plan2.txt
secret/secret-plans

$ oc secrets new-dockercfg my-pull-secret \
    --docker-username=mastermind \
    --docker-password=12345 \
    --docker-email=mastermind@example.com
secret/my-pull-secret

$ oc secrets link robot secret-plans --for=mount

$ oc secrets link robot my-pull-secret --for=pull

$ oc describe serviceaccount robot
Name:               robot
Labels:             <none>
Image pull secrets:	robot-dockercfg-624cx
                   	my-pull-secret

Mountable secrets: 	robot-token-uzkbh
                   	robot-dockercfg-624cx
                   	secret-plans

Tokens:            	robot-token-8bhpp
                   	robot-token-uzkbh

12.7. Using a Service Account’s Credentials Inside a Container

When a pod is created, it specifies a service account (or uses the default service account), and is allowed to use that service account’s API credentials and referenced secrets.

A file containing an API token for a pod’s service account is automatically mounted at /var/run/secrets/kubernetes.io/serviceaccount/token.

That token can be used to make API calls as the pod’s service account. This example calls the users/~ API to get information about the user identified by the token:

$ TOKEN="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"

$ curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
    "https://openshift.default.svc.cluster.local/oapi/v1/users/~" \
    -H "Authorization: Bearer $TOKEN"

kind: "User"
apiVersion: "v1"
metadata:
  name: "system:serviceaccount:top-secret:robot"
  selflink: "/oapi/v1/users/system:serviceaccount:top-secret:robot"
  creationTimestamp: null
identities: null
groups:
  - "system:serviceaccounts"
  - "system:serviceaccounts:top-secret"

12.8. Using a Service Account’s Credentials Externally

The same token can be distributed to external applications that need to authenticate to the API.

Use the following syntax to to view a service account’s API token:

$ oc describe secret <secret-name>

For example:

$ oc describe secret robot-token-uzkbh -n top-secret
Name:		robot-token-uzkbh
Labels:		<none>
Annotations:	kubernetes.io/service-account.name=robot,kubernetes.io/service-account.uid=49f19e2e-16c6-11e5-afdc-3c970e4b7ffe

Type:	kubernetes.io/service-account-token

Data

token:	eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...

$ oc login --token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...
Logged into "https://server:8443" as "system:serviceaccount:top-secret:robot" using the token provided.

You don't have any projects. You can try to create a new project, by running

    $ oc new-project <projectname>

$ oc whoami
system:serviceaccount:top-secret:robot

Chapter 13. Managing Images

13.1. Overview

An image stream comprises any number of container images identified by tags. It presents a single virtual view of related images, similar to a Docker image repository.

By watching an image stream, builds and deployments can receive notifications when new images are added or modified and react by performing a build or deployment, respectively.

There are many ways you can interact with images and set up image streams, depending on where the images' registries are located, any authentication requirements around those registries, and how you want your builds and deployments to behave. The following sections cover a range of these topics.

13.2. Tagging Images

Before working with OpenShift Online image streams and their tags, it will help to first understand image tags in the context of Docker generally.

Container images can have names added to them that make it more intuitive to determine what they contain, called a tag. Using a tag to specify the version of what is contained in the image is a common use case. If you have an image named ruby, you could have a tag named 2.0 for 2.0 version of Ruby, and another named latest to indicate literally the latest built image in that repository overall.

When interacting directly with images using the docker CLI, the docker tag command can add tags, which essentially adds an alias to an image that can consist of several parts. Those parts can include:

<registry_server>/<user_name>/<image_name>:<tag>

The <user_name> part in the above could also refer to a project or namespace if the image is being stored in an OpenShift Online environment with an internal registry.

OpenShift Online provides the oc tag command, which is similar to the docker tag command, but operates on image streams instead of directly on images.

Note

See Red Hat Enterprise Linux 7’s Getting Started with Containers documentation for more about tagging images directly using the docker CLI.

13.2.1. Adding Tags to Image Streams

Keeping in mind that an image stream in OpenShift Online comprises zero or more container images identified by tags, you can add tags to an image stream using the oc tag command:

$ oc tag <source> <destination>

For example, to configure the ruby image streams static-2.0 tag to always refer to the current image for the ruby image streams 2.0 tag:

$ oc tag ruby:2.0 ruby:static-2.0

This will create a new image stream tag named static-2.0 in the ruby image stream. The new tag will directly reference the image id that the ruby:2.0 image stream tag pointed to at the time oc tag was run, and the image it points to will never change.

There are different types of tags available. The default behavior uses a permanent tag, which points to a specific image in time; even when the source changes, the new (destination) tag will not change.

A tracking tag means the destination tag’s metadata will be updated during the import of the source tag. To ensure the destination tag is updated whenever the source tag changes, use the --alias=true flag:

$ oc tag --alias=true <source> <destination>
Note

Use a tracking tag for creating permanent aliases (for example, latest or stable). The tag works correctly only within a single image stream. Trying to create a cross-image-stream alias will produce an error.

You can also add the --scheduled=true flag to have the destination tag be refreshed (i.e., re-imported) periodically. The period is configured globally at system level. See Importing Tag and Image Metadata for more details.

The --reference flag will create an image stream tag that is not imported. The tag will simply point to the source location, permanently.

If you want to instruct Docker to always fetch the tagged image from the integrated registry, use --reference-policy=local. The registry uses the pull-through feature to serve the image to the client. By default, the image blobs are mirrored locally by the registry. As a result, they can be pulled more quickly the next time they are needed. The flag also allows for pulling from insecure registries without a need to supply --insecure-registry to the Docker daemon as long as the image stream has an insecure annotation or the tag has an insecure import policy.

13.2.2. Tag Naming

Images evolve over time and the tag reflects this. It always points to the latest image built. If there is too much information embedded in a tag name (for example, v2.0.1-may-2016), the tag will point to just one revision of an image and will never be updated. Using default image pruning options, such an image will never be removed. Instead, if the tag is named v2.0, more image revisions are more likely. This results in longer tag history and, therefore, the image pruner will more likely remove old and unused images.

Although tag naming convention is up to you, here are a few examples:

DescriptionExample

Revision

v2.0.1

Architecture

v2.0-x86_64

Base image

v1.2-centos7

If you require dates in tag names, periodically inspect old and unsupported images and istags and remove them. Otherwise, you might experience increasing resource usage caused by old images.

13.2.3. Removing Tags from Image Streams

To remove a tag completely from an image stream run:

$ oc delete istag/ruby:latest

or:

$ oc tag -d ruby:latest

13.2.4. Referencing Images in Image Streams

Images can be referenced in image streams using the following reference types:

  • An ImageStreamTag is used to reference or retrieve an image for a given image stream and tag. It uses the following convention for its name:

    <image_stream_name>:<tag>
  • An ImageStreamImage is used to reference or retrieve an image for a given image stream and image name. It uses the following convention for its name:

    <image_stream_name>@<id>

    The <id> is an immutable identifier for a specific image, also called a digest.

  • A DockerImage is used to reference or retrieve an image for a given external registry. It uses standard Docker pull specification for its name, e.g.:

    openshift/ruby-20-centos7:2.0
    Note

    When no tag is specified, it is assumed the latest tag will be used.

    You can also reference a third-party registry:

    registry.access.redhat.com/rhel7:latest

    Or an image with a digest:

    centos/ruby-22-centos7@sha256:3a335d7d8a452970c5b4054ad7118ff134b3a6b50a2bb6d0c07c746e8986b28e

When viewing example image stream definitions, such as the example CentOS image streams, you may notice they contain definitions of ImageStreamTag and references to DockerImage, but nothing related to ImageStreamImage.

This is because the ImageStreamImage objects are automatically created in OpenShift Online whenever you import or tag an image into the image stream. You should never have to explicitly define an ImageStreamImage object in any image stream definition that you use to create image streams.

You can view an image’s object definition by retrieving an ImageStreamImage definition using the image stream name and ID:

$ oc export isimage <image_stream_name>@<id>
Note

You can find valid <id> values for a given image stream by running:

$ oc describe is <image_stream_name>

For example, from the ruby image stream asking for the ImageStreamImage with the name and ID of ruby@3a335d7:

Example 13.1. Definition of an Image Object Retrieved via ImageStreamImage

$ oc export isimage ruby@3a335d7

apiVersion: v1
image:
  dockerImageLayers:
  - name: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
    size: 0
  - name: sha256:ee1dd2cb6df21971f4af6de0f1d7782b81fb63156801cfde2bb47b4247c23c29
    size: 196634330
  - name: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
    size: 0
  - name: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
    size: 0
  - name: sha256:ca062656bff07f18bff46be00f40cfbb069687ec124ac0aa038fd676cfaea092
    size: 177723024
  - name: sha256:63d529c59c92843c395befd065de516ee9ed4995549f8218eac6ff088bfa6b6e
    size: 55679776
  dockerImageMetadata:
    Architecture: amd64
    Author: SoftwareCollections.org <sclorg@redhat.com>
    Config:
      Cmd:
      - /bin/sh
      - -c
      - $STI_SCRIPTS_PATH/usage
      Entrypoint:
      - container-entrypoint
      Env:
      - PATH=/opt/app-root/src/bin:/opt/app-root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
      - STI_SCRIPTS_URL=image:///usr/libexec/s2i
      - STI_SCRIPTS_PATH=/usr/libexec/s2i
      - HOME=/opt/app-root/src
      - BASH_ENV=/opt/app-root/etc/scl_enable
      - ENV=/opt/app-root/etc/scl_enable
      - PROMPT_COMMAND=. /opt/app-root/etc/scl_enable
      - RUBY_VERSION=2.2
      ExposedPorts:
        8080/tcp: {}
      Image: d9c3abc5456a9461954ff0de8ae25e0e016aad35700594714d42b687564b1f51
      Labels:
        build-date: 2015-12-23
        io.k8s.description: Platform for building and running Ruby 2.2 applications
        io.k8s.display-name: Ruby 2.2
        io.openshift.builder-base-version: 8d95148
        io.openshift.builder-version: 8847438ba06307f86ac877465eadc835201241df
        io.openshift.expose-services: 8080:http
        io.openshift.s2i.scripts-url: image:///usr/libexec/s2i
        io.openshift.tags: builder,ruby,ruby22
        io.s2i.scripts-url: image:///usr/libexec/s2i
        license: GPLv2
        name: CentOS Base Image
        vendor: CentOS
      User: "1001"
      WorkingDir: /opt/app-root/src
    ContainerConfig: {}
    Created: 2016-01-26T21:07:27Z
    DockerVersion: 1.8.2-el7
    Id: 57b08d979c86f4500dc8cad639c9518744c8dd39447c055a3517dc9c18d6fccd
    Parent: d9c3abc5456a9461954ff0de8ae25e0e016aad35700594714d42b687564b1f51
    Size: 430037130
    apiVersion: "1.0"
    kind: DockerImage
  dockerImageMetadataVersion: "1.0"
  dockerImageReference: centos/ruby-22-centos7@sha256:3a335d7d8a452970c5b4054ad7118ff134b3a6b50a2bb6d0c07c746e8986b28e
  metadata:
    creationTimestamp: 2016-01-29T13:17:45Z
    name: sha256:3a335d7d8a452970c5b4054ad7118ff134b3a6b50a2bb6d0c07c746e8986b28e
    resourceVersion: "352"
    uid: af2e7a0c-c68a-11e5-8a99-525400f25e34
kind: ImageStreamImage
metadata:
  creationTimestamp: null
  name: ruby@3a335d7
  namespace: openshift
  selflink: /oapi/v1/namespaces/openshift/imagestreamimages/ruby@3a335d7

13.3. Using Image Streams with Kubernetes Resources (Technology Preview)

Image Streams, being OpenShift Online native resources, work out of the box with all the rest of native resources available in OpenShift Online, such as builds or deployments. Currently, it is also possible to make them work with native Kubernetes resources, such as jobs, replication controllers, replica sets or Kubernetes deployments. The cluster administrator configures exactly what resources] can be used.

Kubernetes deployments.

When enabled, it is possible to put a reference to an image stream in the image field of a resource. When using this feature, it is only possible to reference image streams that reside in the same project as the resource. The image stream reference must consist of a single segment value, for example ruby:2.4, where ruby is the name of an image stream that has a tag named 2.4 and resides in the same project as the resource making the reference.

There are two ways to enable this:

  1. Enabling image stream resolution on a specific resource. This will allow only this resource to use the image stream name in the image field.
  2. Enabling image stream resolution on an image stream. This will allow all resources pointing to this image stream to use it in the image field.

Both of these operations can be done using oc set image-lookup. For example, the following command allows all resources to reference the image stream named mysql:

$ oc set image-lookup mysql

This will set the Imagestream.spec.lookupPolicy.local field to true.

Example 13.2. Image stream with image lookup enabled

apiVersion: v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/display-name: mysql
  name: mysql
  namespace: myproject
spec:
  lookupPolicy:
    local: true

When enabled, the behavior is enabled for all tags within the image stream.

You can query the image streams and see if the option is set using:

$ oc set image-lookup

You can also enable image lookup on a specific resource. This command allows the Kubernetes deployment named mysql to use image streams:

$ oc set image-lookup deploy/mysql

This will set the alpha.image.policy.openshift.io/resolve-names annotation on the deployment.

Example 13.3. Deployment with image lookup enabled

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mysql
  namespace: myproject
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        alpha.image.policy.openshift.io/resolve-names: '*'
    spec:
      containers:
      - image: mysql:latest
        imagePullPolicy: Always
        name: mysql

To disable image lookup, pass --enabled=false:

$ oc set image-lookup deploy/mysql --enabled=false

13.4. Image Pull Policy

Each container in a pod has a container image. Once you have created an image and pushed it to a registry, you can then refer to it in the pod.

When OpenShift Online creates containers, it uses the container’s imagePullPolicy to determine if the image should be pulled prior to starting the container. There are three possible values for imagePullPolicy:

  • Always - always pull the image.
  • IfNotPresent - only pull the image if it does not already exist on the node.
  • Never - never pull the image.

If a container’s imagePullPolicy parameter is not specified, OpenShift Online sets it based on the image’s tag:

  1. If the tag is latest, OpenShift Online defaults imagePullPolicy to Always.
  2. Otherwise, OpenShift Online defaults imagePullPolicy to IfNotPresent.

13.5. Accessing the Internal Registry

You can access OpenShift Online’s internal registry directly to push or pull images. For example, this could be helpful if you wanted to create an image stream by manually pushing an image, or just to docker pull an image directly.

OpenShift Online gives developers a hands-on preview of the OpenShift platform in a hosted environment that includes access to an internal registry.

The internal registry authenticates using the same tokens as the OpenShift Online API. To perform a docker login against the internal registry, you can choose any user name and email, but the password must be a valid OpenShift Online token.

To log into the internal registry:

  1. Log in to OpenShift Online:

    $ oc login
  2. Get your access token:

    $ oc whoami -t
  3. Log in to the internal registry using the token. You must have docker installed on your system:

    $ docker login -u <user_name> -e <email_address> \
        -p <token_value> https://registry.<clusterID>.openshift.com
    Note

    Contact your cluster administrator if you do not know the registry IP or host name and port to use.

In order to pull an image, the authenticated user must have get rights on the requested imagestreams/layers. In order to push an image, the authenticated user must have update rights on the requested imagestreams/layers.

By default, all service accounts in a project have rights to pull any image in the same project, and the builder service account has rights to push any image in the same project.

13.6. Using Image Pull Secrets

Docker registries can be secured to prevent unauthorized parties from accessing certain images. If you are using OpenShift Online’s internal registry and are pulling from image streams located in the same project, then your pod’s service account should already have the correct permissions and no additional action should be required.

However, for other scenarios, such as referencing images across OpenShift Online projects or from secured registries, then additional configuration steps are required. The following sections detail these scenarios and their required steps.

13.6.1. Allowing Pods to Reference Images Across Projects

When using the internal registry, to allow pods in project-a to reference images in project-b, a service account in project-a must be bound to the system:image-puller role in project-b:

$ oc policy add-role-to-user \
    system:image-puller system:serviceaccount:project-a:default \
    --namespace=project-b

After adding that role, the pods in project-a that reference the default service account will be able to pull images from project-b.

To allow access for any service account in project-a, use the group:

$ oc policy add-role-to-group \
    system:image-puller system:serviceaccounts:project-a \
    --namespace=project-b

13.6.2. Allowing Pods to Reference Images from Other Secured Registries

The .dockercfg file (or $HOME/.docker/config.json for newer Docker clients) is a Docker credentials file that stores your information if you have previously logged into a secured or insecure registry.

To pull a secured container image that is not from OpenShift Online’s internal registry, you must create a pull secret from your Docker credentials and add it to your service account.

If you already have a .dockercfg file for the secured registry, you can create a secret from that file by running:

$ oc secrets new <pull_secret_name> .dockercfg=<path/to/.dockercfg>

Or if you have a $HOME/.docker/config.json file:

$ oc secrets new <pull_secret_name> .dockerconfigjson=<path/to/.docker/config.json>

If you do not already have a Docker credentials file for the secured registry, you can create a secret by running:

$ oc secrets new-dockercfg <pull_secret_name> \
    --docker-server=<registry_server> --docker-username=<user_name> \
    --docker-password=<password> --docker-email=<email>

To use a secret for pulling images for pods, you must add the secret to your service account. The name of the service account in this example should match the name of the service account the pod will use; default is the default service account:

$ oc secrets link default <pull_secret_name> --for=pull

To use a secret for pushing and pulling build images, the secret must be mountable inside of a pod. You can do this by running:

$ oc secrets link builder <pull_secret_name>

13.6.2.1. Pulling from Private Registries with Delegated Authentication

A private registry can delegate authentication to a separate service. In these cases, image pull secrets must be defined for both the authentication and registry endpoints.

Note

Third-party images in the Red Hat Container Catalog are served from the Red Hat Connect Partner Registry (registry.connect.redhat.com). This registry delegates authentication to sso.redhat.com, so the following procedure applies.

  1. Create a secret for the delegated authentication server:

    $ oc secret new-dockercfg \
        --docker-server=sso.redhat.com \
        --docker-username=developer@example.com \
        --docker-password=******** \
        --docker-email=unused \
        redhat-connect-sso
    
    secret/redhat-connect-sso
  2. Create a secret for the private registry:

    $ oc secret new-dockercfg \
        --docker-server=privateregistry.example.com \
        --docker-username=developer@example.com \
        --docker-password=******** \
        --docker-email=unused \
        private-registry
    
    secret/private-registry
Note

The Red Hat Connect Partner Registry (registry.connect.redhat.com) does not accept the auto-generated dockercfg secret type (BZ#1476330). A generic file-based secret must be created using the generated file from a docker login command:

$ docker login registry.connect.redhat.com --username developer@example.com

Password: *************
Login Succeeded

$ oc secret new redhat-connect .dockerconfigjson=/root/.docker/config.json

$ oc secrets link default redhat-connect --for=pull

13.7. Importing Tag and Image Metadata

An image stream can be configured to import tag and image metadata from an image repository in an external Docker image registry. You can do this using a few different methods.

  • You can manually import tag and image information with the oc import-image command using the --from option:

    $ oc import-image <image_stream_name>[:<tag>] --from=<docker_image_repo> --confirm

    For example:

    $ oc import-image my-ruby --from=docker.io/openshift/ruby-20-centos7 --confirm
    The import completed successfully.
    
    Name:			my-ruby
    Created:		Less than a second ago
    Labels:			<none>
    Annotations:		openshift.io/image.dockerRepositoryCheck=2016-05-06T20:59:30Z
    Docker Pull Spec:	172.30.94.234:5000/demo-project/my-ruby
    
    Tag	Spec					Created			PullSpec							Image
    latest	docker.io/openshift/ruby-20-centos7	Less than a second ago	docker.io/openshift/ruby-20-centos7@sha256:772c5bf9b2d1e8...	<same>

    You can also add the --all flag to import all tags for the image instead of just latest.

  • Like most objects in OpenShift Online, you can also write and save a JSON or YAML definition to a file then create the object using the CLI. Set the spec.dockerImageRepository field to the Docker pull spec for the image:

    apiVersion: "v1"
    kind: "ImageStream"
    metadata:
      name: "my-ruby"
    spec:
      dockerImageRepository: "docker.io/openshift/ruby-20-centos7"

    Then create the object:

    $ oc create -f <file>

When you create an image stream that references an image in an external Docker registry, OpenShift Online communicates with the external registry within a short amount of time to get up to date information about the image.

After the tag and image metadata is synchronized, the image stream object would look similar to the following:

apiVersion: v1
kind: ImageStream
metadata:
  name: my-ruby
  namespace: demo-project
  selflink: /oapi/v1/namespaces/demo-project/imagestreams/my-ruby
  uid: 5b9bd745-13d2-11e6-9a86-0ada84b8265d
  resourceVersion: '4699413'
  generation: 2
  creationTimestamp: '2016-05-06T21:34:48Z'
  annotations:
    openshift.io/image.dockerRepositoryCheck: '2016-05-06T21:34:48Z'
spec:
  dockerImageRepository: docker.io/openshift/ruby-20-centos7
  tags:
    -
      name: latest
      annotations: null
      from:
        kind: DockerImage
        name: 'docker.io/openshift/ruby-20-centos7:latest'
      generation: 2
      importPolicy: {  }
status:
  dockerImageRepository: '172.30.94.234:5000/demo-project/my-ruby'
  tags:
    -
      tag: latest
      items:
        -
          created: '2016-05-06T21:34:48Z'
          dockerImageReference: 'docker.io/openshift/ruby-20-centos7@sha256:772c5bf9b2d1e8e80742ed75aab05820419dc4532fa6d7ad8a1efddda5493dc3'
          image: 'sha256:772c5bf9b2d1e8e80742ed75aab05820419dc4532fa6d7ad8a1efddda5493dc3'
          generation: 2

You can set a tag to query external registries at a scheduled interval to synchronize tag and image metadata by setting the --scheduled=true flag with the oc tag command as mentioned in Adding Tags to Image Streams.

Alternatively, you can set importPolicy.scheduled to true in the tag’s definition:

apiVersion: v1
kind: ImageStream
metadata:
  name: ruby
spec:
  tags:
  - from:
      kind: DockerImage
      name: openshift/ruby-20-centos7
    name: latest
    importPolicy:
      scheduled: true

13.7.1. Importing Images from Insecure Registries

An image stream can be configured to import tag and image metadata from insecure image registries, such as those signed with a self-signed certificate or using plain HTTP instead of HTTPS.

To configure this, add the openshift.io/image.insecureRepository annotation and set it to true. This setting bypasses certificate validation when connecting to the registry:

kind: ImageStream
apiVersion: v1
metadata:
  name: ruby
  annotations:
    openshift.io/image.insecureRepository: "true" 1
  spec:
    dockerImageRepository: my.repo.com:5000/myimage
1
Set the openshift.io/image.insecureRepository annotation to true
Important

This option instructs integrated registry to fall back to an insecure transport for any external image tagged in the image stream when serving it, which is dangerous. If possible, avoid this risk by marking just an istag as insecure.

13.7.1.1. Image Stream Tag Policies

13.7.1.1.1. Insecure Tag Import Policy

The above annotation applies to all images and tags of a particular ImageStream. For a finer-grained control, policies may be set on istags. Set importPolicy.insecure in the tag’s definition to true to allow a fall-back to insecure transport just for images under this tag.

Note

The fall-back to insecure transport for an image under particular istag will be enabled either when the image stream is annotated as insecure or the istag has insecure import policy. The importPolicy.insecure` set to false can not override the image stream annotation.

13.7.1.1.2. Reference Policy

The Reference Policy allows you to specify from where resources that reference this image stream tag will pull the image. It is only applicable to remote images (those imported from external registries). There are two options to choose from, Local and Source.

The Source policy instructs clients to pull directly from the source registry of the image. The integrated registry is not involved unless the image is managed by the cluster. (It is not an external image.) This is the default policy.

The Local policy instructs clients to always pull from the integrated registry. This is useful if you want to pull from external insecure registries without modifying Docker daemon settings.

This policy only affects the use of the image stream tag. Components or operations that directly reference or pull the image using its external registry location will not be redirected to the internal registry.

The pull-through feature of the registry serves the remote image to the client. This feature, which is on by default, must be enabled for the local reference policy to be used. Additionally, by default, all the blobs are mirrored for faster access later.

You can set the policy in a specification of image stream tag as referencePolicy.type.

Example 13.4. Exmple of Insecure Tag with a Local Reference Policy

kind: ImageStream
apiVersion: v1
metadata:
  name: ruby
  tags:
  - from:
      kind: DockerImage
      name: my.repo.com:5000/myimage
    name: mytag
    importPolicy:
      insecure: true 1
    referencePolicy:
      type: Local 2
1
Set tag mytag to use an insecure connection to that registry.
2
Set tag mytag to use integrated registry for pulling external images. If the reference policy type is set to Source, clients fetch the image directly from my.repo.com:5000/myimage.

13.7.2. Importing Images from Private Registries

An image stream can be configured to import tag and image metadata from private image registries, requiring authentication.

To configure this, you need to create a secret which is used to store your credentials.

Create the secret first, before importing the image from the private repository:

$ oc secrets new-dockercfg <secret_name> \
    --docker-server=<docker_registry_server> \
    --docker-username=<docker_user> \
    --docker-password=<docker_password> \
    --docker-email=<docker_email>

For more options, see:

$ oc secrets new --help

After the secret is configured, proceed with creating the new image stream or using the oc import-image command. During the import process, OpenShift Online will pick up the secrets and provide them to the remote party.

13.7.3. Adding Trusted Certificates for External Registries

If the registry you are importing from is using a certificate that is not signed by a standard certificate authority, you will need to explicitly configure the system to trust the registry’s certificate or signing authority. This can be done by adding the CA certificate or registry certificate to the host system running the registry import controller (typically the master node).

The certificate or CA certificate must be added to /etc/pki/tls/certs or /etc/pki/ca-trust, respectively, on the host system. The update-ca-trust command will also need to be run on Red Hat distributions followed by a restart of the master service to pick up the certificate changes.

13.7.4. Importing Images Across Projects

An image stream can be configured to import tag and image metadata from the internal registry, but from a different project. The recommended method for this is to use the oc tag command as shown in Adding Tags to Image Streams:

$ oc tag <source_project>/<image_stream>:<tag> <new_image_stream>:<new_tag>

Another method is to import the image from the other project manually using the pull spec:

Warning

The following method is strongly discouraged and should be used only if the former using oc tag is insufficient.

  1. First, add the necessary policy to access the other project:

    $ oc policy add-role-to-group \
        system:image-puller \
        system:serviceaccounts:<destination_project> \
        -n <source_project>

    This allows <destination_project> to pull images from <source_project>.

  2. With the policy in place, you can import the image manually:

    $ oc import-image <new_image_stream> --confirm \
        --from=<docker_registry>/<source_project>/<image_stream>

13.7.5. Creating an Image Stream by Manually Pushing an Image

An image stream can also be automatically created by manually pushing an image to the internal registry. This is only possible when using an OpenShift Online internal registry.

Before performing this procedure, the following must be satisfied:

  • The destination project you push to must already exist.
  • The user must be authorized to {get, update} "imagestream/layers" in that project. In addition, since the image stream does not already exist, the user must be authorized to {create} "imagestream" in that project. If you are a project administrator, then you would have these permissions.
Note

The system:image-pusher role does not grant permission to create new image streams, only to push images to existing image streams, so it cannot be used to push images to image streams that do not yet exist unless additional permissions are also granted to the user.

To create an image stream by manually pushing an image:

  1. First, log in to the internal registry.
  2. Then, tag your image using the appropriate internal registry location. For example, if you had already pulled the docker.io/centos:centos7 image locally:

    $ docker tag docker.io/centos:centos7 172.30.48.125:5000/test/my-image
  3. Finally, push the image to your internal registry. For example:

    $ docker push 172.30.48.125:5000/test/my-image
    The push refers to a repository [172.30.48.125:5000/test/my-image] (len: 1)
    c8a648134623: Pushed
    2bf4902415e3: Pushed
    latest: digest: sha256:be8bc4068b2f60cf274fc216e4caba6aa845fff5fa29139e6e7497bb57e48d67 size: 6273
  4. Verify that the image stream was created:

    $ oc get is
    NAME       DOCKER REPO                        TAGS      UPDATED
    my-image   172.30.48.125:5000/test/my-image   latest    3 seconds ago

13.8. Writing Image Streams for S2I Builders

Image streams for S2I builders that are displayed in the management console’s catalog page require additional metadata to provide the best experience for end users.

Example 13.5. Definition of an Image Stream Object with Catalog Metadata

apiVersion: v1
kind: ImageStream
metadata:
  name: ruby
  annotations:
    openshift.io/display-name: Ruby 1
spec:
  tags:
    - name: '2.0' 2
      annotations:
        openshift.io/display-name: Ruby 2.0 3
        description: >- 4
          Build and run Ruby 2.0 applications on CentOS 7. For more information
          about using this builder image, including OpenShift considerations,
          see
          https://github.com/sclorg/s2i-ruby-container/tree/master/2.0/README.md.
        iconClass: icon-ruby 5
        sampleRepo: 'https://github.com/openshift/ruby-ex.git' 6
        tags: 'builder,ruby' 7
        supports: 'ruby' 8
        version: '2.0' 9
      from:
        kind: DockerImage
        name: 'openshift/ruby-20-centos7:latest'
1
A brief, user-friendly name for the whole image stream.
2
The tag is referred to as the version. Tags appear in a drop-down menu.
3
A user-friendly name for this tag within the image stream. This should be brief and include version information when appropriate.
4
A description of the tag, which includes enough detail for users to understand what the image is providing. It can include links to additional instructions. Limit the description to a few sentences.
5
The icon to show for this tag. Pick from our existing logo icons when possible. Icons from FontAwesome and Patternfly can also be used. Alternatively, provide icons through CSS customizations that can be added to an OpenShift Online cluster that uses your image stream. You must specify an icon class that exists, or it will prevent falling back to the generic icon.
6
A URL to a source repository that works with this builder image tag and results in a sample running application.
7
Categories that the image stream tag is associated with. The builder tag is required for it to show up in the catalog. Add tags that will associate it with one of the provided catalog categories. Refer to the id and categoryAliases in CATALOG_CATEGORIES in the console’s constants file. The categories can also be customized for the whole cluster.
8
Languages this image supports. This value will be used during oc new-app invocations to try to match potential builder images to the provided source repository.
9
Version information for this tag.

Chapter 14. Quotas and Limit Ranges

14.1. Overview

Using quotas and limit ranges, cluster administrators can set constraints to limit the number of objects or amount of compute resources that are used in your project. This helps cluster administrators better manage and allocate resources across all projects, and ensure that no projects are using more than is appropriate for the cluster size.

As a developer, you can also set requests and limits on compute resources at the pod and container level.

The following sections help you understand how to check on your quota and limit range settings, what sorts of things they can constrain, and how you can request or limit compute resources in your own pods and containers.

14.2. Quotas

A resource quota, defined by a ResourceQuota object, provides constraints that limit aggregate resource consumption per project. It can limit the quantity of objects that can be created in a project by type, as well as the total amount of compute resources that may be consumed by resources in that project.

Note

Quotas are set by cluster administrators and are scoped to a given project.

14.2.1. Viewing Quotas

You can view usage statistics related to any hard limits defined in a project’s quota by navigating in the web console to the project’s Settings tab.

You can also use the CLI to view quota details:

  1. First, get the list of quotas defined in the project. For example, for a project called demoproject:

    $ oc get quota -n demoproject
    NAME                AGE
    besteffort          11m
    compute-resources   2m
    core-object-counts  29m
  2. Then, describe the quota you are interested in, for example the core-object-counts quota:

    $ oc describe quota core-object-counts -n demoproject
    Name:			core-object-counts
    Namespace:		demoproject
    Resource		Used	Hard
    --------		----	----
    configmaps		3	10
    persistentvolumeclaims	0	4
    replicationcontrollers	3	20
    secrets			9	10
    services		2	10

Full quota definitions can be viewed by running oc export on the object. The following show some sample quota definitions:

Example 14.1. core-object-counts.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: core-object-counts
spec:
  hard:
    configmaps: "10" 1
    persistentvolumeclaims: "4" 2
    replicationcontrollers: "20" 3
    secrets: "10" 4
    services: "10" 5
1
The total number of ConfigMap objects that can exist in the project.
2
The total number of persistent volume claims (PVCs) that can exist in the project.
3
The total number of replication controllers that can exist in the project.
4
The total number of secrets that can exist in the project.
5
The total number of services that can exist in the project.

Example 14.2. openshift-object-counts.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: openshift-object-counts
spec:
  hard:
    openshift.io/imagestreams: "10" 1
1
The total number of image streams that can exist in the project.

Example 14.3. compute-resources.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-resources
spec:
  hard:
    pods: "4" 1
    requests.cpu: "1" 2
    requests.memory: 1Gi 3
    limits.cpu: "2" 4
    limits.memory: 2Gi 5
1
The total number of pods in a non-terminal state that can exist in the project.
2
Across all pods in a non-terminal state, the sum of CPU requests cannot exceed 1 core.
3
Across all pods in a non-terminal state, the sum of memory requests cannot exceed 1Gi.
4
Across all pods in a non-terminal state, the sum of CPU limits cannot exceed 2 cores.
5
Across all pods in a non-terminal state, the sum of memory limits cannot exceed 2Gi.

Example 14.4. besteffort.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: besteffort
spec:
  hard:
    pods: "1" 1
  scopes:
  - BestEffort 2
1
The total number of pods in a non-terminal state with BestEffort quality of service that can exist in the project.
2
Restricts the quota to only matching pods that have BestEffort quality of service for either memory or CPU.

14.2.2. Resources Managed by Quota

The following describes the set of compute resources and object types that may be managed by a quota.

Table 14.1. Compute Resources Managed by Quota

Resource NameDescription

cpu

Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value.

memory

Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value.

requests.cpu

Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value.

requests.memory

Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value.

limits.cpu

Across all pods in a non-terminal state, the sum of CPU limits cannot exceed this value.

limits.memory

Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value.

Table 14.2. Object Counts Managed by Quota

Resource NameDescription

pods

The total number of pods in a non-terminal state that can exist in the project. A pod is in a terminal state if status.phase in (Failed, Succeeded) is true.

replicationcontrollers

The total number of replication controllers that can exist in the project.

resourcequotas

The total number of resource quotas that can exist in the project.

services

The total number of services that can exist in the project.

secrets

The total number of secrets that can exist in the project.

configmaps

The total number of ConfigMap objects that can exist in the project.

persistentvolumeclaims

The total number of persistent volume claims that can exist in the project.

openshift.io/imagestreams

The total number of image streams that can exist in the project.

14.2.3. Quota Scopes

Each quota can have an associated set of scopes. A quota will only measure usage for a resource if it matches the intersection of enumerated scopes.

When a scope is added to a quota, it limits the number of resources it supports to those that pertain to the scope. Resources specified on the quota outside of the allowed set results in a validation error.

ScopeDescription

Terminating

Match pods where spec.activeDeadlineSeconds >= 0

NotTerminating

Match pods where spec.activeDeadlineSeconds is nil

BestEffort

Match pods that have best effort quality of service for either cpu or memory.

NotBestEffort

Match pods that do not have best effort quality of service for cpu and memory.

A BestEffort scope restricts a quota to limit the following resources:

  • pods

A Terminating, NotTerminating, and NotBestEffort scope restricts a quota to tracking the following resources:

  • pods
  • memory
  • requests.memory
  • limits.memory
  • cpu
  • requests.cpu
  • limits.cpu

14.2.4. Quota Enforcement

After a resource quota for a project is first created, the project restricts the ability to create any new resources that may violate a quota constraint until it has calculated updated usage statistics.

After a quota is created and usage statistics are updated, the project accepts the creation of new content. When you create or modify resources, your quota usage is incremented immediately upon the request to create or modify the resource.

When you delete a resource, your quota use is decremented during the next full recalculation of quota statistics for the project. If project modifications exceed a quota usage limit, the server denies the action. An appropriate error message is returned explaining the quota constraint violated, and what your currently observed usage stats are in the system.

14.2.5. Requests vs Limits

When allocating compute resources, each container may specify a request and a limit value for either CPU or memory. The quota can be configured to quota either value.

If the quota has a value specified for requests.cpu or requests.memory, then it requires that every incoming container makes an explicit request for those resources. If the quota has a value specified for limits.cpu or limits.memory, then it requires that every incoming container specifies an explicit limit for those resources.

See Compute Resources for more on setting requests and limits in pods and containers.

14.3. Limit Ranges

A limit range, defined by a LimitRange object, enumerates compute resource constraints in a project at the pod, container, image, image stream, and persistent volume claim level, and specifies the amount of resources that a pod, container, image, image stream, or persistent volume claim can consume.

All resource create and modification requests are evaluated against each LimitRange object in the project. If the resource violates any of the enumerated constraints, then the resource is rejected. If the resource does not set an explicit value, and if the constraint supports a default value, then the default value is applied to the resource.

Note

Limit ranges are set by cluster administrators and are scoped to a given project.

14.3.1. Viewing Limit Ranges

You can view any limit ranges defined in a project by navigating in the web console to the project’s Settings tab.

You can also use the CLI to view limit range details:

  1. First, get the list of limit ranges defined in the project. For example, for a project called demoproject:

    $ oc get limits -n demoproject
    NAME              AGE
    resource-limits   6d
  2. Then, describe the limit range you are interested in, for example the resource-limits limit range:

    $ oc describe limits resource-limits
    Name:                     limits
    Namespace:                default
    Type                      Resource                 Min  Max Request Limit Limit/Request
    ----                      --------                 ---  --- ------- ----- -------------
    Pod                       memory                   6Mi  1Gi -       -     -
    Pod                       cpu                      200m  2  -       -     -
    Container                 cpu                      100m  2  200m    300m  10
    Container                 memory                   4Mi  1Gi 100Mi   200Mi -
    openshift.io/Image        storage                  -    1Gi -       -     -
    openshift.io/ImageStream  openshift.io/image-tags  -    10  -       -     -
    openshift.io/ImageStream  openshift.io/images      -    12  -       -     -

Full limit range definitions can be viewed by running oc export on the object. The following shows an example limit range definition:

Example 14.5. Core Limit Range Object Definition

apiVersion: "v1"
kind: "LimitRange"
metadata:
  name: "core-resource-limits" 1
spec:
  limits:
    - type: "Pod"
      max:
        cpu: "2" 2
        memory: "1Gi" 3
      min:
        cpu: "200m" 4
        memory: "6Mi" 5
    - type: "Container"
      max:
        cpu: "2" 6
        memory: "1Gi" 7
      min:
        cpu: "100m" 8
        memory: "4Mi" 9
      default:
        cpu: "300m" 10
        memory: "200Mi" 11
      defaultRequest:
        cpu: "200m" 12
        memory: "100Mi" 13
      maxLimitRequestRatio:
        cpu: "10" 14
1
The name of the limit range document.
2
The maximum amount of CPU that a pod can request on a node across all containers.
3
The maximum amount of memory that a pod can request on a node across all containers.
4
The minimum amount of CPU that a pod can request on a node across all containers.
5
The minimum amount of memory that a pod can request on a node across all containers.
6
The maximum amount of CPU that a single container in a pod can request.
7
The maximum amount of memory that a single container in a pod can request.
8
The minimum amount of CPU that a single container in a pod can request.
9
The minimum amount of memory that a single container in a pod can request.
10
The default amount of CPU that a container will be limited to use if not specified.
11
The default amount of memory that a container will be limited to use if not specified.
12
The default amount of CPU that a container will request to use if not specified.
13
The default amount of memory that a container will request to use if not specified.
14
The maximum amount of CPU burst that a container can make as a ratio of its limit over request.

Example 14.6. OpenShift Online Limit Range Object Definition

apiVersion: "v1"
kind: "LimitRange"
metadata:
  name: "openshift-resource-limits"
spec:
  limits:
    - type: openshift.io/Image
      max:
        storage: 1Gi 1
    - type: openshift.io/ImageStream
      max:
        openshift.io/image-tags: 20 2
        openshift.io/images: 30 3
1
The maximum size of an image that can be pushed to an internal registry.
2
The maximum number of unique image tags per image stream’s spec.
3
The maximum number of unique image references per image stream’s status.

Both core and OpenShift Online resources can be specified in just one limit range object. They are separated here into two examples for clarity.

14.3.2. Container Limits

Supported Resources:

  • CPU
  • Memory

Supported Constraints:

Per container, the following must hold true if specified:

Table 14.3. Container

ConstraintBehavior

Min

Min[resource] less than or equal to container.resources.requests[resource] (required) less than or equal to container/resources.limits[resource] (optional)

If the configuration defines a min CPU, then the request value must be greater than the CPU value. A limit value does not need to be specified.

Max

container.resources.limits[resource] (required) less than or equal to Max[resource]

If the configuration defines a max CPU, then you do not need to define a request value, but a limit value does need to be set that satisfies the maximum CPU constraint.

MaxLimitRequestRatio

MaxLimitRequestRatio[resource] less than or equal to ( container.resources.limits[resource] / container.resources.requests[resource])

If a configuration defines a maxLimitRequestRatio value, then any new containers must have both a request and limit value. Additionally, OpenShift Online calculates a limit to request ratio by dividing the limit by the request.

For example, if a container has cpu: 500 in the limit value, and cpu: 100 in the request value, then its limit to request ratio for cpu is 5. This ratio must be less than or equal to the maxLimitRequestRatio.

Supported Defaults:

Default[resource]
Defaults container.resources.limit[resource] to specified value if none.
Default Requests[resource]
Defaults container.resources.requests[resource] to specified value if none.

14.3.3. Pod Limits

Supported Resources:

  • CPU
  • Memory

Supported Constraints:

Across all containers in a pod, the following must hold true:

Table 14.4. Pod

ConstraintEnforced Behavior

Min

Min[resource] less than or equal to container.resources.requests[resource] (required) less than or equal to container.resources.limits[resource] (optional)

Max

container.resources.limits[resource] (required) less than or equal to Max[resource]

MaxLimitRequestRatio

MaxLimitRequestRatio[resource] less than or equal to ( container.resources.limits[resource] / container.resources.requests[resource])

14.4. Compute Resources

Each container running on a node consumes compute resources, which are measurable quantities that can be requested, allocated, and consumed.

When authoring a pod configuration file, you can optionally specify how much CPU and memory (RAM) each container needs in order to better schedule pods in the cluster and ensure satisfactory performance.

CPU is measured in units called millicores. Each node in a cluster inspects the operating system to determine the amount of CPU cores on the node, then multiplies that value by 1000 to express its total capacity. For example, if a node has 2 cores, the node’s CPU capacity would be represented as 2000m. If you wanted to use 1/10 of a single core, it would be represented as 100m.

Memory is measured in bytes. In addition, it may be used with SI suffices (E, P, T, G, M, K) or their power-of-two-equivalents (Ei, Pi, Ti, Gi, Mi, Ki).

apiVersion: v1
kind: Pod
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        cpu: 100m 1
        memory: 200Mi 2
      limits:
        cpu: 200m 3
        memory: 400Mi 4
1
The container requests 100m cpu.
2
The container requests 200Mi memory.
3
The container limits 200m cpu.
4
The container limits 400Mi memory.

14.4.1. CPU Requests

Each container in a pod can specify the amount of CPU it requests on a node. The scheduler uses CPU requests to find a node with an appropriate fit for a container.

The CPU request represents a minimum amount of CPU that your container may consume, but if there is no contention for CPU, it can use all available CPU on the node. If there is CPU contention on the node, CPU requests provide a relative weight across all containers on the system for how much CPU time the container may use.

On the node, CPU requests map to Kernel CFS shares to enforce this behavior.

Note

In OpenShift Online, CPU requests are set automatically based on the memory limit specified. If no memory limit is specified, a CPU request of 60m is set.

14.4.2. Viewing Compute Resources

To view compute resources for a pod:

$ oc describe pod nginx-tfjxt
Name:       nginx-tfjxt
Namespace:      default
Image(s):     nginx
Node:       /
Labels:       run=nginx
Status:       Pending
Reason:
Message:
IP:
Replication Controllers:  nginx (1/1 replicas created)
Containers:
  nginx:
    Container ID:
    Image:    nginx
    Image ID:
    QoS Tier:
      cpu:  Burstable
      memory: Burstable
    Limits:
      cpu:  200m
      memory: 400Mi
    Requests:
      cpu:    100m
      memory:   200Mi
    State:    Waiting
    Ready:    False
    Restart Count:  0
    Environment Variables:

14.4.3. CPU Limits

Each container in a pod can specify the amount of CPU it is limited to use on a node. CPU limits control the maximum amount of CPU that your container may use independent of contention on the node. If a container attempts to exceed the specified limit, the system will throttle the container. This allows the container to have a consistent level of service independent of the number of pods scheduled to the node.

Note

In OpenShift Online, CPU limits are set automatically based on the memory limit specified. If no memory limit is specified, a CPU limit of 1 core is set.

14.4.4. Memory Requests

By default, a container is able to consume as much memory on the node as possible. In order to improve placement of pods in the cluster, specify the amount of memory required for a container to run. The scheduler will then take available node memory capacity into account prior to binding your pod to a node. A container is still able to consume as much memory on the node as possible even when specifying a request.

Note

In OpenShift Online, memory requests are set automatically based on the memory limit specified. If no memory limit is specified, a memory request of 307Mi is assumed.

14.4.5. Memory Limits

If you specify a memory limit, you can constrain the amount of memory the container can use. For example, if you specify a limit of 200Mi, a container will be limited to using that amount of memory on the node. If the container exceeds the specified memory limit, it will be terminated and potentially restarted dependent upon the container restart policy.

Note

In OpenShift Online, the memory request, CPU request, and CPU limit will automatically be determined and set appropriately based off of the specified memory limit. If no memory limit is specified, a memory limit of 512Mi is assumed.

14.4.6. Quality of Service Tiers

A compute resource is classified with a quality of service (QoS) based on the specified request and limit value.

Quality of ServiceDescription

BestEffort

Provided when a request and limit are not specified.

Burstable

Provided when a request is specified that is less than an optionally specified limit.

Guaranteed

Provided when a limit is specified that is equal to an optionally specified request.

A container may have a different quality of service for each compute resource. For example, a container can have Burstable CPU and Guaranteed memory qualities of service.

The quality of service has different impacts on different resources, depending on whether the resource is compressible or not. CPU is a compressible resource, whereas memory is an incompressible resource.

With CPU Resources:
  • A BestEffort CPU container is able to consume as much CPU as is available on a node but runs with the lowest priority.
  • A Burstable CPU container is guaranteed to get the minimum amount of CPU requested, but it may or may not get additional CPU time. Excess CPU resources are distributed based on the amount requested across all containers on the node.
  • A Guaranteed CPU container is guaranteed to get the amount requested and no more, even if there are additional CPU cycles available. This provides a consistent level of performance independent of other activity on the node.
With Memory Resources:
  • A BestEffort memory container is able to consume as much memory as is available on the node, but there are no guarantees that the scheduler will place that container on a node with enough memory to meet its needs. In addition, a BestEffort container has the greatest chance of being killed if there is an out of memory event on the node.
  • A Burstable memory container is scheduled on the node to get the amount of memory requested, but it may consume more. If there is an out of memory event on the node, Burstable containers are killed after BestEffort containers when attempting to recover memory.
  • A Guaranteed memory container gets the amount of memory requested, but no more. In the event of an out of memory event, it will only be killed if there are no more BestEffort or Burstable containers on the system.

14.4.7. Specifying Compute Resources via CLI

To specify compute resources via the CLI:

$ oc run nginx --image=nginx --limits=memory=400Mi

Chapter 15. Routes

15.1. Overview

An OpenShift Online route exposes a service at a host name, like www.example.com, so that external clients can reach it by name.

DNS resolution for a host name is handled separately from routing; your administrator may have configured a cloud domain that will always correctly resolve to the OpenShift Online router, or if using an unrelated host name you may need to modify its DNS records independently to resolve to the router.

15.2. Creating Routes

You can create unsecured and secured routes routes using the web console or the CLI.

Using the web console, you can navigate to the Browse → Routes page, then click Create Route to define and create a route in your project:

Figure 15.1. Creating a Route Using the Web Console

Creating a Route Using the Web Console

Using the CLI, create an unsecured route. For OpenShift Online Starter, follow this example:

$ oc expose svc/frontend

For OpenShift Online Pro, follow this example, with --hostname being optional:

$ oc expose svc/frontend --hostname=www.example.com

The new route inherits the name from the service unless you specify one using the --name option.

YAML Definition of the Unsecured Route Created Above

apiVersion: v1
kind: Route
metadata:
  name: frontend
spec:
  to:
    kind: Service
    name: frontend

Unsecured routes are the default configuration, and are therefore the simplest to set up. However, secured routes offer security for connections to remain private. To create a secured HTTPS route encrypted with the default certificate for OpenShift Online 3 you can use the create route command.

Note

TLS is the replacement of SSL for HTTPS and other encrypted protocols.

For OpenShift Online Starter:

$ oc create route edge --service=frontend

YAML Definition of the Secured Route Created Above

apiVersion: v1
kind: Route
metadata:
  name: frontend
spec:
  to:
    kind: Service
    name: frontend
  tls:
    termination: edge

For OpenShift Online Pro, you can use your own certificate and key files from a CA. However, you can still omit the certificate and key files if you want to use the default certificate. With OpenShift Online Starter, you cannot specify a certificate and key.

For OpenShift Online Pro:

$ oc create route edge --service=frontend \
    --cert=example.crt \
    --key=example.key \
    --ca-cert=ca.crt \
    --hostname=www.example.com

YAML Definition of the Secured Route Created Above

apiVersion: v1
kind: Route
metadata:
  name: frontend
spec:
  host: www.example.com
  to:
    kind: Service
    name: frontend
  tls:
    termination: edge
    key: |-
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----

Currently, password protected key files are not supported. To remove a passphrase from a keyfile, you can run:

# openssl rsa -in <passwordProtectedKey.key> -out <new.key>

Further information on all types of TLS termination as well as path-based routing are available in the Architecture section.

15.3. Restrictions

Routes are restricted in OpenShift Online Starter, but are not restricted in OpenShift Online Pro. Custom route hosts are permitted in OpenShift Online Pro. If using OpenShift Online Starter, the following host template is enforced on all user routes:

<route-name>-<namespace>.<external-address>

For example:

<route-name>-<namespace>.1d35.starter-us-east-1.openshiftapps.com

To determine the external address, run:

$ oc get route/<route-name>

Custom certificates are permitted in OpenShift Online Pro. In OpenShift Online Starter, only unencrypted routes, edge routes using the default certificate, and passthrough routes work. Edge and re-encrypt routes with custom certificates do not work in OpenShift Online Starter.

These restrictions are enforced by the API. Attempts to create routes with custom hosts or certificates will be rejected in OpenShift Online Starter. In OpenShift Online Pro, a default host is provided if the user does not specify a custom host.

Once your custom route is created in in OpenShift Online Pro, you must update your DNS provider by creating a canonical name (CNAME) record. Your CNAME record should point your custom domain to the OpenShift Online router as the alias. The OpenShift Online router’s domain is different for every cluster.

Note

CNAME records cannot be set for a naked domain (example.com). A subdomain must be specified (www.example.com).

Chapter 16. Integrating External Services

16.1. Overview

Many OpenShift Online applications use external resources, such as external databases, or an external SaaS endpoint. These external resources can be modeled as native OpenShift Online services, so that applications can work with them as they would any other internal service.

Egress traffic can be controlled by firewall rules or an Egress router. This permits having a static IP address for their application service.

16.2. Defining a Service for an External Database

One of the most common types of external services is an external database. To support an external database, an application needs:

  1. An endpoint to communicate with.
  2. A set of credentials and coordinates, including:

    1. A user name
    2. A passphrase
    3. A database name

The solution for integrating with an external database includes:

  • A Service object to represent the SaaS provider as an OpenShift Online service.
  • One or more Endpoints for the service.
  • Environment variables in the appropriate pods containing the credentials.

The following steps outline a scenario for integrating with an external MySQL database:

16.2.1. Step 1: Define a Service

You can define a service either by providing an IP address and endpoints, or by providing a Fully qualified domain name (FQDN).

16.2.1.1. Using an IP address

  1. Create an OpenShift Online service to represent your external database. This is similar to creating an internal service; the difference is in the service’s Selector field.

    Internal OpenShift Online services use the Selector field to associate pods with services using labels. The EndpointsController system component synchronizes the endpoints for services that specify selectors with the pods that match the selector. The service proxy and OpenShift Online router load-balance requests to the service amongst the service’s endpoints.

    Services that represent an external resource do not require associated pods. Instead, leave the Selector field unset. This represents the external service, making the EndpointsController ignore the service and allows you to specify endpoints manually:

      kind: "Service"
      apiVersion: "v1"
      metadata:
        name: "external-mysql-service"
      spec:
        ports:
          -
            name: "mysql"
            protocol: "TCP"
            port: 3306
            targetPort: 3306
            nodePort: 0
      selector: {} 1
    1
    The selector field to leave blank.
  2. Next, create the required endpoints for the service. This gives the service proxy and router the location to send traffic directed to the service:

      kind: "Endpoints"
      apiVersion: "v1"
      metadata:
        name: "external-mysql-service" 1
      subsets: 2
        -
          addresses:
            -
              ip: "10.0.0.0" 3
          ports:
            -
              port: 3306 4
              name: "mysql"
    1
    The name of the Service instance, as defined in the previous step.
    2
    Traffic to the service will be load-balanced between the supplied Endpoints if more than one is supplied.
    3
    Endpoints IPs cannot be loopback (127.0.0.0/8), link-local (169.254.0.0/16), or link-local multicast (224.0.0.0/24).
    4
    The port and name definition must match the port and name value in the service defined in the previous step.

16.2.1.2. Using an External Domain Name

Using external domain names make it easier to manage an external service linkage, because you do not have to worry about the external service’s IP addresses changing.

ExternalName services do not have selectors, or any defined ports or endpoints, therefore, you can use an ExternalName service to direct traffic to an external service.

kind: "Service"
apiVersion: "v1"
metadata:
  name: "external-mysql-service"
spec:
  type: ExternalName
  externalName: example.domain.name
selector: {} 1
1
The selector field to leave blank.

Using an external domain name service tells the system that the DNS name in the externalName field (example.domain.name in the previous example) is the location of the resource that backs the service. When a DNS request is made against the Kubernetes DNS server, it returns the externalName in a CNAME record telling the client to look up the returned name to get the IP address.

16.2.2. Step 2: Consume a Service

  1. Now that the service and endpoints are defined, give the appropriate pods access to the credentials to use the service by setting environment variables in the appropriate containers:

    kind: "DeploymentConfig"
    apiVersion: "v1"
    metadata:
      name: "my-app-deployment"
    spec: 1
      strategy:
        type: "Rolling"
        rollingParams:
          updatePeriodSeconds: 1 2
          intervalSeconds: 1 3
          timeoutSeconds: 120
      replicas: 2
      selector:
        name: "frontend"
      template:
        metadata:
          labels:
            name: "frontend"
        spec:
          containers:
            -
              name: "helloworld"
              image: "origin-ruby-sample"
              ports:
                -
                  containerPort: 3306
                  protocol: "TCP"
              env:
                -
                  name: "MYSQL_USER"
                  value: "${MYSQL_USER}" 4
                -
                  name: "MYSQL_PASSWORD"
                  value: "${MYSQL_PASSWORD}" 5
                -
                  name: "MYSQL_DATABASE"
                  value: "${MYSQL_DATABASE}" 6
    1
    Other fields on the DeploymentConfig are omitted
    2
    The user name to use with the service.
    3
    The passphrase to use with the service.
    4
    The database name.

External Database Environment Variables

Using an external service in your application is similar to using an internal service. Your application will be assigned environment variables for the service and the additional environment variables with the credentials described in the previous step. For example, a MySQL container receives the following environment variables:

  • EXTERNAL_MYSQL_SERVICE_SERVICE_HOST=<ip_address>
  • EXTERNAL_MYSQL_SERVICE_SERVICE_PORT=<port_number>
  • MYSQL_USERNAME=<mysql_username>
  • MYSQL_PASSPHRASE=<mysql_passphrase>
  • MYSQL_DATABASE_NAME=<mysql_database>

The application is responsible for reading the coordinates and credentials for the service from the environment and establishing a connection with the database via the service.

16.3. External SaaS Provider

A common type of external service is an external SaaS endpoint. To support an external SaaS provider, an application needs:

  1. An endpoint to communicate with
  2. A set of credentials, such as:

    1. An API key
    2. A user name
    3. A passphrase

The following steps outline a scenario for integrating with an external SaaS provider:

  1. Create an OpenShift Online service to represent the external service. This is similar to creating an internal service; however the difference is in the service’s Selector field.

    Internal OpenShift Online services use the Selector field to associate pods with services using labels. A system component called EndpointsController synchronizes the endpoints for services that specify selectors with the pods that match the selector. The service proxy and OpenShift Online router load-balance requests to the service amongst the service’s endpoints.

    Services that represents an external resource do not require that pods be associated with it. Instead, leave the Selector field unset. This makes the EndpointsController ignore the service and allows you to specify endpoints manually:

      kind: "Service"
      apiVersion: "v1"
      metadata:
        name: "example-external-service"
      spec:
        ports:
          -
            name: "mysql"
            protocol: "TCP"
            port: 3306
            targetPort: 3306
            nodePort: 0
      selector: {} 1
    1
    The selector field to leave blank.
  2. Next, create endpoints for the service containing the information about where to send traffic directed to the service proxy and the router:

    kind: "Endpoints"
    apiVersion: "v1"
    metadata:
      name: "example-external-service" 1
    subsets: 2
    - addresses:
      - ip: "10.10.1.1"
      ports:
      - name: "mysql"
        port: 3306
    1
    The name of the Service instance.
    2
    Traffic to the service is load-balanced between the subsets supplied here.
  3. Now that the service and endpoints are defined, give pods the credentials to use the service by setting environment variables in the appropriate containers:

      kind: "DeploymentConfig"
      apiVersion: "v1"
      metadata:
        name: "my-app-deployment"
      spec:  1
        strategy:
          type: "Rolling"
          rollingParams:
            updatePeriodSeconds: 1
            intervalSeconds: 1
            timeoutSeconds: 120
        replicas: 1
        selector:
          name: "frontend"
        template:
          metadata:
            labels:
              name: "frontend"
          spec:
            containers:
              -
                name: "helloworld"
                image: "openshift/openshift/origin-ruby-sample"
                ports:
                  -
                    containerPort: 3306
                    protocol: "TCP"
                env:
                  -
                    name: "SAAS_API_KEY" 2
                    value: "<SaaS service API key>"
                  -
                    name: "SAAS_USERNAME" 3
                    value: "<SaaS service user>"
                  -
                    name: "SAAS_PASSPHRASE" 4
                    value: "<SaaS service passphrase>"
    1
    Other fields on the DeploymentConfig are omitted.
    2
    SAAS_API_KEY: The API key to use with the service.
    3
    SAAS_USERNAME: The user name to use with the service.
    4
    SAAS_PASSPHRASE: The passphrase to use with the service.

    These variables get added to the containers as environment variables. Using environment variables allows service-to-service communication and it may or may not require additional parameters such as API keys, user name and password authentication, or certificates.

External SaaS Provider Environment Variables

Similarly, when using an internal service, your application is assigned environment variables for the service and the additional environment variables with the credentials described in the previous steps. In the previous example, the container receives the following environment variables:

  • EXAMPLE_EXTERNAL_SERVICE_SERVICE_HOST=<ip_address>
  • EXAMPLE_EXTERNAL_SERVICE_SERVICE_PORT=<port_number>
  • SAAS_API_KEY=<saas_api_key>
  • SAAS_USERNAME=<saas_username>
  • SAAS_PASSPHRASE=<saas_passphrase>

The application reads the coordinates and credentials for the service from the environment and establishes a connection with the service.

16.3.1. Using an External Domain Name

ExternalName services do not have selectors, or any defined ports or endpoints. You can use an ExternalName service to assign traffic to an external service outside the cluster.

  kind: "Service"
  apiVersion: "v1"
  metadata:
    name: "external-mysql-service"
  spec:
    type: ExternalName
    externalName: example.domain.name
  selector: {} 1
1
The selector field to leave blank.

Using an ExternalName service maps the service to the value of the externalName field (example.domain.name in the previous example), by automatically injecting a CNAME record, mapping the service name directly to an outside DNS address, and bypassing the need for endpoint records.

Chapter 17. Secrets

17.1. Using Secrets

This topic discusses important properties of secrets and provides an overview on how developers can use them.

The Secret object type provides a mechanism to hold sensitive information such as passwords, OpenShift Online client configuration files, dockercfg files, private source repository credentials, and so on. Secrets decouple sensitive content from the pods. You can mount secrets into containers using a volume plug-in or the system can use secrets to perform actions on behalf of a pod.

Example 17.1. YAML Secret Object Definition

apiVersion: v1
kind: Secret
metadata:
  name: test-secret
  namespace: my-namespace
data: 1
  username: dmFsdWUtMQ0K 2
  password: dmFsdWUtMg0KDQo=
stringData: 3
  hostname: myapp.mydomain.com 4
1
The allowable format for the keys in the data field must meet the guidelines in the DNS_SUBDOMAIN value in the Kubernetes identifiers glossary.
2
The value associated with keys in the the data map must be base64 encoded.
3
Entries in the stringData map will have their values converted to base64 and the entry will then be moved to the data map automatically. This field is write-only; the value will only be returned via the data field.
4
The value associated with keys in the the stringData map is made up of plain text strings.

17.1.1. Properties of Secrets

Key properties include:

  • Secret data can be referenced independently from its definition.
  • Secret data volumes are backed by temporary file-storage facilities (tmpfs) and never come to rest on a node.
  • Secret data can be shared within a namespace.

17.1.2. Creating Secrets

You must create a secret before creating the pods that depend on that secret.

When creating secrets:

  • Create a secret object with secret data.
  • Update the pod’s service account to allow the reference to the secret.
  • Create a pod, which consumes the secret as an environment variable or as a file (using a secret volume).

You can use the create command to create a secret object from a JSON or YAML file:

$ oc create -f <filename>

17.1.3. Updating Secrets

When you modify the value of a secret, the value (used by an already running pod) will not dynamically change. To change a secret, you must delete the original pod and create a new pod (perhaps with an identical PodSpec).

Updating a secret follows the same workflow as deploying a new container image. You can use the kubectl rolling-update command.

The resourceVersion value in a secret is not specified when it is referenced. Therefore, if a secret is updated at the same time as pods are starting, then the version of the secret will be used for the pod will not be defined.

Note

Currently, it is not possible to check the resource version of a secret object that was used when a pod was created. It is planned that pods will report this information, so that a controller could restart ones using a old resourceVersion. In the interim, do not update the data of existing secrets, but create new ones with distinct names.

17.2. Secrets in Volumes and Environment Variables

See examples of YAML files with secret data.

After you create a secret, you can:

  1. Create the pod to reference your secret:

    $ oc create -f <your_yaml_file>.yaml
  2. Get the logs:

    $ oc logs secret-example-pod
  3. Delete the pod:

    $ oc delete pod secret-example-pod

17.3. Image Pull Secrets

See Using Image Pull Secrets for more information.

17.4. Source Clone Secrets

See Build Inputs for more information about using source clone secrets during a build.

17.5. Service Serving Certificate Secrets

Service serving certificate secrets are intended to support complex middleware applications that need out-of-the-box certificates. It has the same settings as the server certificates generated by the administrator tooling for nodes and masters.

To secure communication to your service, have the cluster generate a signed serving certificate/key pair into a secret in your namespace. To do this, set the service.alpha.openshift.io/serving-cert-secret-name annotation on your service with the value set to the name you want to use for your secret. Then, your PodSpec can mount that secret. When it is available, your pod will run. The certificate will be good for the internal service DNS name, <service.name>.<service.namespace>.svc.

The certificate and key are in PEM format, stored in tls.crt and tls.key respectively. The certificate/key pair is automatically replaced when it gets close to expiration. View the expiration date in the service.alpha.openshift.io/expiry annotation on the secret, which is in RFC3339 format.

Other pods can trust cluster-created certificates (which are only signed for internal DNS names), by using the CA bundle in the /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt file that is automatically mounted in their pod.

The signature algorithm for this feature is x509.SHA256WithRSA. To manually rotate, delete the generated secret. A new certificate is created.

17.6. Restrictions

To use a secret, a pod needs to reference the secret. A secret can be used with a pod in three ways:

  • to populate environment variables for containers.
  • as files in a volume mounted on one or more of its containers.
  • by kubelet when pulling images for the pod.

Volume type secrets write data into the container as a file using the volume mechanism. imagePullSecrets use service accounts for the automatic injection of the secret into all pods in a namespaces.

When a template contains a secret definition, the only way for the template to use the provided secret is to ensure that the secret volume sources are validated and that the specified object reference actually points to an object of type Secret. Therefore, a secret needs to be created before any pods that depend on it. The most effective way to ensure this is to have it get injected automatically through the use of a service account.

Secret API objects reside in a namespace. They can only be referenced by pods in that same namespace.

Individual secrets are limited to 1MB in size. This is to discourage the creation of large secrets that would exhaust apiserver and kubelet memory. However, creation of a number of smaller secrets could also exhaust memory.

17.6.1. Secret Data Keys

Secret keys must be in a DNS subdomain.

17.7. Examples

Example 17.2. YAML Secret That Will Create Four Files

apiVersion: v1
kind: Secret
metadata:
  name: test-secret
data:
  username: dmFsdWUtMQ0K     1
  password: dmFsdWUtMQ0KDQo= 2
stringData:
  hostname: myapp.mydomain.com 3
  secret.properties: |-     4
    property1=valueA
    property2=valueB
1
File contains decoded values.
2
File contains decoded values.
3
File contains the provided string.
4
File contains the provided data.

Example 17.3. YAML of a Pod Populating Files in a Volume with Secret Data

apiVersion: v1
kind: Pod
metadata:
  name: secret-example-pod
spec:
  containers:
    - name: secret-test-container
      image: busybox
      command: [ "/bin/sh", "-c", "cat /etc/secret-volume/*" ]
      volumeMounts:
          # name must match the volume name below
          - name: secret-volume
            mountPath: /etc/secret-volume
            readOnly: true
  volumes:
    - name: secret-volume
      secret:
        secretName: test-secret
  restartPolicy: Never

Example 17.4. YAML of a Pod Populating Environment Variables with Secret Data

apiVersion: v1
kind: Pod
metadata:
  name: secret-example-pod
spec:
  containers:
    - name: secret-test-container
      image: busybox
      command: [ "/bin/sh", "-c", "export" ]
      env:
        - name: TEST_SECRET_USERNAME_ENV_VAR
          valueFrom:
            secretKeyRef:
              name: test-secret
              key: username
  restartPolicy: Never

17.8. Troubleshooting

If a service certificate generations fails with (service’s service.alpha.openshift.io/serving-cert-generation-error annotation contains):

secret/ssl-key references serviceUID 62ad25ca-d703-11e6-9d6f-0e9c0057b608, which does not match 77b6dd80-d716-11e6-9d6f-0e9c0057b60

The service that generated the certificate no longer exists, or has a different serviceUID. You must force certificates regeneration by removing the old secret, and clearing the following annotations on the service service.alpha.openshift.io/serving-cert-generation-error, service.alpha.openshift.io/serving-cert-generation-error-num:

$ oc delete secret <secret_name>
$ oc annotate service <service_name> service.alpha.openshift.io/serving-cert-generation-error-
$ oc annotate service <service_name> service.alpha.openshift.io/serving-cert-generation-error-num-
Note

The command removing annotation has a - after the annotation name to be removed.

Chapter 18. ConfigMaps

18.1. Overview

Many applications require configuration using some combination of configuration files, command line arguments, and environment variables. These configuration artifacts should be decoupled from image content in order to keep containerized applications portable.

The ConfigMap object provides mechanisms to inject containers with configuration data while keeping containers agnostic of OpenShift Online. A ConfigMap can be used to store fine-grained information like individual properties or coarse-grained information like entire configuration files or JSON blobs.

The ConfigMap API object holds key-value pairs of configuration data that can be consumed in pods or used to store configuration data for system components such as controllers. ConfigMap is similar to secrets, but designed to more conveniently support working with strings that do not contain sensitive information.

For example:

Example 18.1. ConfigMap Object Definition

kind: ConfigMap
apiVersion: v1
metadata:
  creationTimestamp: 2016-02-18T19:14:38Z
  name: example-config
  namespace: default
data: 1
  example.property.1: hello
  example.property.2: world
  example.property.file: |-
    property.1=value-1
    property.2=value-2
    property.3=value-3
1
Contains the configuration data.

Configuration data can be consumed in pods in a variety of ways. A ConfigMap can be used to:

  1. Populate the value of environment variables.
  2. Set command-line arguments in a container.
  3. Populate configuration files in a volume.

Both users and system components may store configuration data in a ConfigMap.

18.2. Creating ConfigMaps

You can use the following command to create a ConfigMap easily from directories, specific files, or literal values:

$ oc create configmap <configmap_name> [options]

The following sections cover the different ways you can create a ConfigMap.

18.2.1. Creating from Directories

Consider a directory with some files that already contain the data with which you want to populate a ConfigMap:

$ ls example-files
game.properties
ui.properties

$ cat example-files/game.properties
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30

$ cat example-files/ui.properties
color.good=purple
color.bad=yellow
allow.textmode=true
how.nice.to.look=fairlyNice

You can use the following command to create a ConfigMap holding the content of each file in this directory:

$ oc create configmap game-config \
    --from-file=example-files/

When the --from-file option points to a directory, each file directly in that directory is used to populate a key in the ConfigMap, where the name of the key is the file name, and the value of the key is the content of the file.

For example, the above command creates the following ConfigMap:

$ oc describe configmaps game-config
Name:           game-config
Namespace:      default
Labels:         <none>
Annotations:    <none>

Data

game.properties:        121 bytes
ui.properties:          83 bytes

You can see the two keys in the map are created from the file names in the directory specified in the command. Because the content of those keys may be large, the output of oc describe only shows the names of the keys and their sizes.

If you want to see the values of the keys, you can oc get the object with the -o option:

$ oc get configmaps game-config -o yaml

apiVersion: v1
data:
  game.properties: |-
    enemies=aliens
    lives=3
    enemies.cheat=true
    enemies.cheat.level=noGoodRotten
    secret.code.passphrase=UUDDLRLRBABAS
    secret.code.allowed=true
    secret.code.lives=30
  ui.properties: |
    color.good=purple
    color.bad=yellow
    allow.textmode=true
    how.nice.to.look=fairlyNice
kind: ConfigMap
metadata:
  creationTimestamp: 2016-02-18T18:34:05Z
  name: game-config
  namespace: default
  resourceVersion: "407"-
  selflink: /api/v1/namespaces/default/configmaps/game-config
  uid: 30944725-d66e-11e5-8cd0-68f728db1985

18.2.2. Creating from Files

You can also pass the --from-file option with a specific file, and pass it multiple times to the CLI. The following yields equivalent results to the Creating from Directories example:

  1. Create the ConfigMap specifying a specific file:

    $ oc create configmap game-config-2 \
        --from-file=example-files/game.properties \
        --from-file=example-files/ui.properties
  2. Verify the results:

    $ oc get configmaps game-config-2 -o yaml
    
    apiVersion: v1
    data:
      game.properties: |-
        enemies=aliens
        lives=3
        enemies.cheat=true
        enemies.cheat.level=noGoodRotten
        secret.code.passphrase=UUDDLRLRBABAS
        secret.code.allowed=true
        secret.code.lives=30
      ui.properties: |
        color.good=purple
        color.bad=yellow
        allow.textmode=true
        how.nice.to.look=fairlyNice
    kind: ConfigMap
    metadata:
      creationTimestamp: 2016-02-18T18:52:05Z
      name: game-config-2
      namespace: default
      resourceVersion: "516"
      selflink: /api/v1/namespaces/default/configmaps/game-config-2
      uid: b4952dc3-d670-11e5-8cd0-68f728db1985

You can also set the key to use for an individual file with the --from-file option by passing an expression of key=value. For example:

  1. Create the ConfigMap specifying a key-value pair:

    $ oc create configmap game-config-3 \
        --from-file=game-special-key=example-files/game.properties
  2. Verify the results:

    $ oc get configmaps game-config-3 -o yaml
    
    apiVersion: v1
    data:
      game-special-key: |-
        enemies=aliens
        lives=3
        enemies.cheat=true
        enemies.cheat.level=noGoodRotten
        secret.code.passphrase=UUDDLRLRBABAS
        secret.code.allowed=true
        secret.code.lives=30
    kind: ConfigMap
    metadata:
      creationTimestamp: 2016-02-18T18:54:22Z
      name: game-config-3
      namespace: default
      resourceVersion: "530"
      selflink: /api/v1/namespaces/default/configmaps/game-config-3
      uid: 05f8da22-d671-11e5-8cd0-68f728db1985

18.2.3. Creating from Literal Values

You can also supply literal values for a ConfigMap. The --from-literal option takes a key=value syntax that allows literal values to be supplied directly on the command line:

  1. Create the ConfigMap specifying a literal value:

    $ oc create configmap special-config \
        --from-literal=special.how=very \
        --from-literal=special.type=charm
  2. Verify the results:

    $ oc get configmaps special-config -o yaml
    
    apiVersion: v1
    data:
      special.how: very
      special.type: charm
    kind: ConfigMap
    metadata:
      creationTimestamp: 2016-02-18T19:14:38Z
      name: special-config
      namespace: default
      resourceVersion: "651"
      selflink: /api/v1/namespaces/default/configmaps/special-config
      uid: dadce046-d673-11e5-8cd0-68f728db1985

18.3. Use Cases: Consuming ConfigMaps in Pods

The following sections describe some uses cases when consuming ConfigMap objects in pods.

18.3.1. Consuming in Environment Variables

A ConfigMap can be used to populate the value of command line arguments. For example, consider the following ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  special.how: very
  special.type: charm

You can consume the keys of this ConfigMap in a pod using configMapKeyRef sections:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: gcr.io/google_containers/busybox
      command: [ "/bin/sh", "-c", "env" ]
      env:
        - name: SPECIAL_LEVEL_KEY
          valueFrom:
            configMapKeyRef:
              name: special-config
              key: special.how
        - name: SPECIAL_TYPE_KEY
          valueFrom:
            configMapKeyRef:
              name: special-config
              key: special.type
  restartPolicy: Never

When this pod is run, its output will include the following lines:

SPECIAL_LEVEL_KEY=very
SPECIAL_TYPE_KEY=charm

18.3.2. Setting Command-line Arguments

A ConfigMap can also be used to set the value of the command or arguments in a container. This is accomplished using the Kubernetes substitution syntax $(VAR_NAME). Consider the following ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  special.how: very
  special.type: charm

To inject values into the command line, you must consume the keys you want to use as environment variables, as in the Consuming in Environment Variables use case. Then you can refer to them in a container’s command using the $(VAR_NAME) syntax.

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: gcr.io/google_containers/busybox
      command: [ "/bin/sh", "-c", "echo $(SPECIAL_LEVEL_KEY) $(SPECIAL_TYPE_KEY)" ]
      env:
        - name: SPECIAL_LEVEL_KEY
          valueFrom:
            configMapKeyRef:
              name: special-config
              key: special.how
        - name: SPECIAL_TYPE_KEY
          valueFrom:
            configMapKeyRef:
              name: special-config
              key: special.type
  restartPolicy: Never

When this pod is run, the output from the test-container container will be:

very charm

18.3.3. Consuming in Volumes

A ConfigMap can also be consumed in volumes. Returning again to the following example ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  special.how: very
  special.type: charm

You have a couple different options for consuming this ConfigMap in a volume. The most basic way is to populate the volume with files where the key is the file name and the content of the file is the value of the key:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: gcr.io/google_containers/busybox
      command: [ "/bin/sh", "cat", "/etc/config/special.how" ]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
  restartPolicy: Never

When this pod is run, the output will be:

very

You can also control the paths within the volume where ConfigMap keys are projected:

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: gcr.io/google_containers/busybox
      command: [ "/bin/sh", "cat", "/etc/config/path/to/special-key" ]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
        items:
        - key: special.how
          path: path/to/special-key
  restartPolicy: Never

When this pod is run, the output will be:

very

18.4. Example: Configuring Redis

For a real-world example, you can configure Redis using a ConfigMap. To inject Redis with the recommended configuration for using Redis as a cache, the Redis configuration file should contain the following:

maxmemory 2mb
maxmemory-policy allkeys-lru

If your configuration file is located at example-files/redis/redis-config, create a ConfigMap with it:

  1. Create the ConfigMap specifying the configuration file:

    $ oc create configmap example-redis-config \
        --from-file=example-files/redis/redis-config
  2. Verify the results:

    $ oc get configmap example-redis-config -o yaml
    
    apiVersion: v1
    data:
      redis-config: |
        maxmemory 2mb
        maxmemory-policy allkeys-lru
    kind: ConfigMap
    metadata:
      creationTimestamp: 2016-04-06T05:53:07Z
      name: example-redis-config
      namespace: default
      resourceVersion: "2985"
      selflink: /api/v1/namespaces/default/configmaps/example-redis-config
      uid: d65739c1-fbbb-11e5-8a72-68f728db1985

Now, create a pod that uses this ConfigMap:

  1. Create a pod definition like the following and save it to a file, for example redis-pod.yaml:

    apiVersion: v1
    kind: Pod
    metadata:
      name: redis
    spec:
      containers:
      - name: redis
        image: kubernetes/redis:v1
        env:
        - name: MASTER
          value: "true"
        ports:
        - containerPort: 6379
        resources:
          limits:
            cpu: "0.1"
        volumeMounts:
        - mountPath: /redis-master-data
          name: data
        - mountPath: /redis-master
          name: config
      volumes:
        - name: data
          emptyDir: {}
        - name: config
          configMap:
            name: example-redis-config
            items:
            - key: redis-config
              path: redis.conf
  2. Create the pod:

    $ oc create -f redis-pod.yaml

The newly-created pod has a ConfigMap volume that places the redis-config key of the example-redis-config ConfigMap into a file called redis.conf. This volume is mounted into the /redis-master directory in the Redis container, placing our configuration file at /redis-master/redis.conf, which is where the image looks for the Redis configuration file for the master.

If you oc exec into this pod and run the redis-cli tool, you can check that the configuration was applied correctly:

$ oc exec -it redis redis-cli
127.0.0.1:6379> CONFIG GET maxmemory
1) "maxmemory"
2) "2097152"
127.0.0.1:6379> CONFIG GET maxmemory-policy
1) "maxmemory-policy"
2) "allkeys-lru"

18.5. Restrictions

A ConfigMap must be created before they are consumed in pods. Controllers can be written to tolerate missing configuration data; consult individual components configured via ConfigMap on a case-by-case basis.

ConfigMap objects reside in a project. They can only be referenced by pods in the same project.

The Kubelet only supports use of a ConfigMap for pods it gets from the API server. This includes any pods created using the CLI, or indirectly from a replication controller. It does not include pods created using the OpenShift Online node’s --manifest-url flag, its --config flag, or its REST API (these are not common ways to create pods).

Chapter 19. Pod Autoscaling

19.1. Overview

A horizontal pod autoscaler, defined by a HorizontalPodAutoscaler object, specifies how the system should automatically increase or decrease the scale of a replication controller or deployment configuration, based on metrics collected from the pods that belong to that replication controller or deployment configuration.

19.2. Supported Metrics

The following metrics are supported by horizontal pod autoscalers:

Table 19.1. Metrics

MetricDescription

CPU utilization

Percentage of the requested CPU

Memory utilization

Percentage of the requested memory.

19.3. Autoscaling

You can create a horizontal pod autoscaler with the oc autoscale command and specify the minimum and maximum number of pods you want to run, as well as the CPU utilization your pods should target.

After a horizontal pod autoscaler is created, it begins attempting to query Heapster for metrics on the pods. It may take one to two minutes before Heapster obtains the initial metrics.

After metrics are available in Heapster, the horizontal pod autoscaler computes the ratio of the current metric utilization with the desired metric utilization, and scales up or down accordingly. The scaling will occur at a regular interval, but it may take one to two minutes before metrics make their way into Heapster.

For replication controllers, this scaling corresponds directly to the replicas of the replication controller. For deployment configurations, scaling corresponds directly to the replica count of the deployment configuration. Note that autoscaling applies only to the latest deployment in the Complete phase.

19.4. Autoscaling for CPU Utilization

Use the oc autoscale command and specify at least the maximum number of pods you want to run at any given time. You can optionally specify the minimum number of pods and the average CPU utilization your pods should target, otherwise those are given default values from the OpenShift Online server.

For example:

$ oc autoscale dc/frontend --min 1 --max 10 /
  --cpu-percent=80 deploymentconfig "frontend" autoscaled

The above example creates a horizontal pod autoscaler with the following definition when using version one of the horizontal pod autoscaler:

Example 19.1. Horizontal Pod Autoscaler Object Definition

apiVersion: extensions/v1beta1
kind: HorizontalPodAutoscaler
metadata:
  name: frontend 1
spec:
  scaleRef:
    kind: DeploymentConfig 2
    name: frontend 3
    apiVersion: v1 4
    subresource: scale
  minReplicas: 1 5
  maxReplicas: 10 6
  cpuUtilization:
    targetPercentage: 80 7
1
The name of this horizontal pod autoscaler object
2
The kind of object to scale
3
The name of the object to scale
4
The API version of the object to scale
5
The minimum number of replicas to which to scale down
6
The maximum number of replicas to which to scale up
7
The percentage of the requested CPU that each pod should ideally be using

Alternatively, the oc autoscale command creates a horizontal pod autoscaler with the following definition when using version two of the horizontal pod autoscaler:

apiVersion: autoscaling/v2alpha1
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-resource-metrics-cpu 1
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1 2
    kind: ReplicationController 3
    name: hello-hpa-cpu 4
  minReplicas: 1 5
  maxReplicas: 10 6
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 50 7
1
The name of this horizontal pod autoscaler object
2
The API version of the object to scale
3
The kind of object to scale
4
The name of the object to scale
5
The minimum number of replicas to which to scale down
6
The maximum number of replicas to which to scale up
7
The average utilization for each pod

19.5. Autoscaling for Memory Utilization

Unlike CPU-based autoscaling, memory-based autoscaling requires specifying the autoscaler using YAML instead of using the oc autoscale command. Optionally, you can specify the minimum number of pods and the average memory utilization your pods should target as well, otherwise those are given default values from the OpenShift Online server.

  1. Memory-based autoscaling is only available with the v2alpha1 version of the autoscaling API. Enable memory-based autoscaling by adding the following to your cluster’s master-config.yaml file:

    ...
    apiServerArguments:
      runtime-config:
      - apis/autoscaling/v2alpha1=true
    ...
  2. Place the following in a file, such as hpa.yaml:

    apiVersion: autoscaling/v2alpha1
    kind: HorizontalPodAutoscaler
    metadata:
      name: hpa-resource-metrics-memory 1
    spec:
      scaleTargetRef:
        apiVersion: apps/v1beta1 2
        kind: ReplicationController 3
        name: hello-hpa-memory 4
      minReplicas: 1 5
      maxReplicas: 10 6
      metrics:
      - type: Resource
        resource:
          name: memory
          targetAverageUtilization: 50 7
    1
    The name of this horizontal pod autoscaler object
    2
    The API version of the object to scale
    3
    The kind of object to scale
    4
    The name of the object to scale
    5
    The minimum number of replicas to which to scale down
    6
    The maximum number of replicas to which to scale up
    7
    The average percentage of the requested memory that each pod should be using
  3. Then, create the autoscaler from the above file:

    $ oc create -f hpa.yaml

19.6. Viewing a Horizontal Pod Autoscaler

To view the status of a horizontal pod autoscaler:

$ oc get hpa/frontend
NAME              REFERENCE                                 TARGET    CURRENT   MINPODS        MAXPODS   AGE
frontend          DeploymentConfig/default/frontend/scale   80%       79%       1              10        8d

$ oc describe hpa/frontend
Name:                           frontend
Namespace:                      default
Labels:                         <none>
CreationTimestamp:              Mon, 26 Oct 2015 21:13:47 -0400
Reference:                      DeploymentConfig/default/frontend/scale
Target CPU utilization:         80%
Current CPU utilization:        79%
Min pods:                       1
Max pods:                       10

Chapter 20. Managing Volumes

20.1. Overview

Containers are not persistent by default; on restart, their contents are cleared. Volumes are mounted file systems available to pods and their containers which may be backed by a number of host-local or network attached storage endpoints.

To ensure that the file system on the volume contains no errors and, if errors are present, to repair them when possible, OpenShift Online invokes the fsck utility prior to the mount utility. This occurs when either adding a volume or updating an existing volume.

The simplest volume type is emptyDir, which is a temporary directory on a single machine. Administrators may also allow you to request a persistent volume that is automatically attached to your pods.

Note

emptyDir volume storage may be restricted by a quota based on the pod’s FSGroup, if the FSGroup parameter is enabled by your cluster administrator.

You can use the CLI command oc volume to add, update, or remove volumes and volume mounts for any object that has a pod template like replication controllers or deployment configurations. You can also list volumes in pods or any object that has a pod template.

20.2. General CLI Usage

The oc volume command uses the following general syntax:

$ oc volume <object_selection> <operation> <mandatory_parameters> <optional_parameters>

This topic uses the form <object_type>/<name> for <object_selection> in later examples. However, you can choose one of the following options:

Table 20.1. Object Selection

SyntaxDescriptionExample

<object_type> <name>

Selects <name> of type <object_type>.

deploymentConfig registry

<object_type>/<name>

Selects <name> of type <object_type>.

deploymentConfig/registry

<object_type>--selector=<object_label_selector>

Selects resources of type <object_type> that matched the given label selector.

deploymentConfig--selector="name=registry"

<object_type> --all

Selects all resources of type <object_type>.

deploymentConfig --all

-f or --filename=<file_name>

File name, directory, or URL to file to use to edit the resource.

-f registry-deployment-config.json

The <operation> can be one of --add, --remove, or --list.

Any <mandatory_parameters> or <optional_parameters> are specific to the selected operation and are discussed in later sections.

20.3. Adding Volumes

To add a volume, a volume mount, or both to pod templates:

$ oc volume <object_type>/<name> --add [options]

Table 20.2. Supported Options for Adding Volumes

OptionDescriptionDefault

--name

Name of the volume.

Automatically generated, if not specified.

-t, --type

Name of the volume source. Supported values: emptyDir, hostPath, secret, configmap, or persistentVolumeClaim.

emptyDir

-c, --containers

Select containers by name. It can also take wildcard '*' that matches any character.

'*'

-m, --mount-path

Mount path inside the selected containers.

 

--path

Host path. Mandatory parameter for --type=hostPath.

 

--secret-name

Name of the secret. Mandatory parameter for --type=secret.

 

--configmap-name

Name of the configmap. Mandatory parameter for --type=configmap.

 

--claim-name

Name of the persistent volume claim. Mandatory parameter for --type=persistentVolumeClaim.

 

--source

Details of volume source as a JSON string. Recommended if the desired volume source is not supported by --type. See available volume sources

 

-o, --output

Display the modified objects instead of updating them on the server. Supported values: json, yaml.

 

--output-version

Output the modified objects with the given version.

api-version

Examples

Add a new volume source emptyDir to deployment configuration registry:

$ oc volume dc/registry --add

Add volume v1 with secret $ecret for replication controller r1 and mount inside the containers at /data:

$ oc volume rc/r1 --add --name=v1 --type=secret --secret-name='$ecret' --mount-path=/data

Add existing persistent volume v1 with claim name pvc1 to deployment configuration dc.json on disk, mount the volume on container c1 at /data, and update the deployment configuration on the server:

$ oc volume -f dc.json --add --name=v1 --type=persistentVolumeClaim \
  --claim-name=pvc1 --mount-path=/data --containers=c1

Add volume v1 based on Git repository https://github.com/namespace1/project1 with revision 5125c45f9f563 for all replication controllers:

$ oc volume rc --all --add --name=v1 \
  --source='{"gitRepo": {
                "repository": "https://github.com/namespace1/project1",
                "revision": "5125c45f9f563"
            }}'

20.4. Updating Volumes

Updating existing volumes or volume mounts is the same as adding volumes, but with the --overwrite option:

$ oc volume <object_type>/<name> --add --overwrite [options]

Examples

Replace existing volume v1 for replication controller r1 with existing persistent volume claim pvc1:

$ oc volume rc/r1 --add --overwrite --name=v1 --type=persistentVolumeClaim --claim-name=pvc1

Change deployment configuration d1 mount point to /opt for volume v1:

$ oc volume dc/d1 --add --overwrite --name=v1 --mount-path=/opt

20.5. Removing Volumes

To remove a volume or volume mount from pod templates:

$ oc volume <object_type>/<name> --remove [options]

Table 20.3. Supported Options for Removing Volumes

OptionDescriptionDefault

--name

Name of the volume.

 

-c, --containers

Select containers by name. It can also take wildcard '*' that matches any character.

'*'

--confirm

Indicate that you want to remove multiple volumes at once.

 

-o, --output

Display the modified objects instead of updating them on the server. Supported values: json, yaml.

 

--output-version

Output the modified objects with the given version.

api-version

Examples

Remove a volume v1 from deployment configuration d1:

$ oc volume dc/d1 --remove --name=v1

Unmount volume v1 from container c1 for deployment configuration d1 and remove the volume v1 if it is not referenced by any containers on d1:

$ oc volume dc/d1 --remove --name=v1 --containers=c1

Remove all volumes for replication controller r1:

$ oc volume rc/r1 --remove --confirm

20.6. Listing Volumes

To list volumes or volume mounts for pods or pod templates:

$ oc volume <object_type>/<name> --list [options]

List volume supported options:

OptionDescriptionDefault

--name

Name of the volume.

 

-c, --containers

Select containers by name. It can also take wildcard '*' that matches any character.

'*'

Examples

List all volumes for pod p1:

$ oc volume pod/p1 --list

List volume v1 defined on all deployment configurations:

$ oc volume dc --all --name=v1

20.7. Specifying a Sub-path

Use the volumeMounts.subPath property to specify a subPath inside a volume instead of the volume’s root. subPath allows you to share one volume for multiple uses in a single pod.

To view the list of files in the volume, run the oc rsh command:

$ oc rsh <pod>
sh-4.2$ ls /path/to/volume/subpath/mount
example_file1 example_file2 example_file3

Specify the subPath:

Example subPath Usage

apiVersion: v1
kind: Pod
metadata:
  name: my-site
spec:
    containers:
    - name: mysql
      image: mysql
      volumeMounts:
      - mountPath: /var/lib/mysql
        name: site-data
        subPath: mysql 1
    - name: php
      image: php
      volumeMounts:
      - mountPath: /var/www/html
        name: site-data
        subPath: html 2
    volumes:
    - name: site-data
      persistentVolumeClaim:
        claimName: my-site-data

1
Databases are stored in the mysql folder.
2
HTML content is stored in the html folder.

Chapter 21. Using Persistent Volumes

21.1. Overview

A PersistentVolume object is a storage resource in an OpenShift Online cluster. Storage is provisioned by your cluster administrator by creating PersistentVolume objects from sources such as GCE Persistent Disk, AWS Elastic Block Store (EBS), and NFS mounts.

Storage can be made available to you by laying claims to the resource. You can make a request for storage resources using a PersistentVolumeClaim object; the claim is paired with a volume that generally matches your request.

21.2. Requesting Storage

You can request storage by creating PersistentVolumeClaim objects in your projects:

Persistent Volume Claim Object Definition

apiVersion: "v1"
kind: "PersistentVolumeClaim"
metadata:
  name: "claim1"
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "1Gi"
  volumeName: "pv0001"

Important

See the Storage topic for information about restrictions when using persistent volumes with OpenShift Online.

21.3. Volume and Claim Binding

A PersistentVolume is a specific resource. A PersistentVolumeClaim is a request for a resource with specific attributes, such as storage size. In between the two is a process that matches a claim to an available volume and binds them together. This allows the claim to be used as a volume in a pod. OpenShift Online finds the volume backing the claim and mounts it into the pod.

You can tell whether a claim or volume is bound by querying using the CLI:

$ oc get pvc
NAME        LABELS    STATUS    VOLUME
claim1      map[]     Bound     pv0001

$ oc get pv
NAME                LABELS              CAPACITY            ACCESSMODES         STATUS    CLAIM
pv0001              map[]               5368709120          RWO                 Bound     yournamespace / claim1

21.4. Claims as Volumes in Pods

A PersistentVolumeClaim is used by a pod as a volume. OpenShift Online finds the claim with the given name in the same namespace as the pod, then uses the claim to find the corresponding volume to mount.

Pod Definition with a Claim

apiVersion: "v1"
kind: "Pod"
metadata:
  name: "mypod"
  labels:
    name: "frontendhttp"
spec:
  containers:
    -
      name: "myfrontend"
      image: "nginx"
      ports:
        -
          containerPort: 80
          name: "http-server"
      volumeMounts:
        -
          mountPath: "/var/www/html"
          name: "pvol"
  volumes:
    -
      name: "pvol"
      persistentVolumeClaim:
        claimName: "claim1"

21.5. Volume and Claim Pre-binding

If you know exactly what PersistentVolume you want your PersistentVolumeClaim to bind to, you can specify the PV in your PVC using the volumeName field. This method skips the normal matching and binding process. The PVC will only be able to bind to a PV that has the same name specified in volumeName. If such a PV with that name exists and is Available, the PV and PVC will be bound regardless of whether the PV satisfies the PVC’s label selector, access modes, and resource requests.

Example 21.1. Persistent Volume Claim Object Definition with volumeName

apiVersion: "v1"
kind: "PersistentVolumeClaim"
metadata:
  name: "claim1"
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "1Gi"
  volumeName: "pv0001"
Important

The ability to set claimRefs is a temporary workaround for the described use cases. A long-term solution for limiting who can claim a volume is in development.

You may also want your cluster administrator to "reserve" the volume for only your claim so that nobody else’s claim can bind to it before yours does. In this case, the administrator can specify the PVC in the PV using the claimRef field. The PV will only be able to bind to a PVC that has the same name and namespace specified in claimRef. The PVC’s access modes and resource requests must still be satisfied in order for the PV and PVC to be bound, though the label selector is ignored.

Persistent Volume Object Definition with claimRef

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0001
spec:
  capacity:
    storage: 1Gi
  accessModes:
  - ReadWriteOnce
  nfs:
    path: /tmp
    server: 172.17.0.2
  persistentVolumeReclaimPolicy: Recycle
  claimRef:
    name: claim1
    namespace: default

Specifying a volumeName in your PVC does not prevent a different PVC from binding to the specified PV before yours does. Your claim will remain Pending until the PV is Available.

Specifying a claimRef in a PV does not prevent the specified PVC from being bound to a different PV. The PVC is free to choose another PV to bind to according to the normal binding process. Therefore, to avoid these scenarios and ensure your claim gets bound to the volume you want, you must ensure that both volumeName and claimRef are specified.

You can tell that your setting of volumeName and/or claimRef influenced the matching and binding process by inspecting a Bound PV and PVC pair for the pv.kubernetes.io/bound-by-controller annotation. The PVs and PVCs where you set the volumeName and/or claimRef yourself will have no such annotation, but ordinary PVs and PVCs will have it set to "yes".

When a PV has its claimRef set to some PVC name and namespace, and is reclaimed according to a Retain or Recycle reclaim policy, its claimRef will remain set to the same PVC name and namespace even if the PVC or the whole namespace no longer exists.

Chapter 22. Storage Classes

22.1. Overview

The StorageClass resource object describes and classifies storage that can be requested, as well as provides a means for passing parameters for dynamically provisioned storage on demand. StorageClass objects can also serve as a management mechanism for controlling different levels of storage and access to the storage. Cluster administrators (users with cluster-admin privileges) or storage administrators (users with storage-admin privileges) define and create the StorageClass objects that users can request without needing any intimate knowledge about the underlying storage volume sources.

In OpenShift Online, the storage class is configured and a single option is available to the user based on the underlying cloud provider.

Chapter 23. Selector and Label Volume Binding

23.1. Overview

You can enable binding of persistent volume claims (PVCs) to persistent volumes (PVs) via selector and label attributes. By implementing selectors and labels, regular users are able to target provisioned storage by identifiers defined by a cluster administrator.

23.2. Motivation

In cases of statically provisioned storage, developers seeking persistent storage are required to know a handful identifying attributes of a PV in order to deploy and bind a PVC. This creates several problematic situations. Regular users might have to contact a cluster administrator to either deploy the PVC or provide the PV values. PV attributes alone do not convey the intended use of the storage volumes, nor do they provide methods by which volumes can be grouped.

Selector and label attributes can be used to abstract away PV details from the user while providing cluster administrators a way of identifying volumes by a descriptive and customizable tag. Through the selector-label method of binding, users are only required to know which labels are defined by the administrator.

Note

The selector-label feature is currently only available for statically provisioned storage and is currently not implemented for storage provisioned dynamically.

Chapter 24. Executing Remote Commands

24.1. Overview

You can use the CLI to execute remote commands in a container. This allows you to run general Linux commands for routine operations in the container.

Important

For security purposes, the oc exec command does not work when accessing privileged containers. See the CLI operations topic for more information.

24.2. Basic Usage

Support for remote container command execution is built into the CLI:

$ oc exec <pod> [-c <container>] <command> [<arg_1> ... <arg_n>]

For example:

$ oc exec mypod date
Thu Apr  9 02:21:53 UTC 2015

24.3. Protocol

Clients initiate the execution of a remote command in a container by issuing a request to the Kubernetes API server:

/proxy/minions/<node_name>/exec/<namespace>/<pod>/<container>?command=<command>

In the above URL:

  • <node_name> is the FQDN of the node.
  • <namespace> is the namespace of the target pod.
  • <pod> is the name of the target pod.
  • <container> is the name of the target container.
  • <command> is the desired command to be executed.

For example:

/proxy/minions/node123.openshift.com/exec/myns/mypod/mycontainer?command=date

Additionally, the client can add parameters to the request to indicate if:

  • the client should send input to the remote container’s command (stdin).
  • the client’s terminal is a TTY.
  • the remote container’s command should send output from stdout to the client.
  • the remote container’s command should send output from stderr to the client.

After sending an exec request to the API server, the client upgrades the connection to one that supports multiplexed streams; the current implementation uses SPDY.

The client creates one stream each for stdin, stdout, and stderr. To distinguish among the streams, the client sets the streamType header on the stream to one of stdin, stdout, or stderr.

The client closes all streams, the upgraded connection, and the underlying connection when it is finished with the remote command execution request.

Note

Administrators can see the Architecture guide for more information.

Chapter 25. Copying Files to or from a Container

25.1. Overview

You can use the CLI to copy local files to or from a remote directory in a container. This is a useful tool for copying database archives to and from your pods for backup and restore purposes. It can also be used to copy source code changes into a running pod for development debugging, when the running pod supports hot reload of source files.

25.2. Basic Usage

Support for copying local files to or from a container is built into the CLI:

$ oc rsync <source> <destination> [-c <container>]

For example, to copy a local directory to a pod directory:

$ oc rsync /home/user/source devpod1234:/src

Or to copy a pod directory to a local directory:

$ oc rsync devpod1234:/src /home/user/source

25.3. Backing Up and Restoring Databases

Use oc rsync to copy database archives from an existing database container to a new database container’s persistent volume directory.

Note

MySQL is used in the example below. Replace mysql|MYSQL with pgsql|PGSQL or mongodb|MONGODB and refer to the migration guide to find the exact commands for each of our supported database images. The example assumes an existing database container.

  1. Back up the existing database from a running database pod:

    $ oc rsh <existing db container>
    # mkdir /var/lib/mysql/data/db_archive_dir
    # mysqldump --skip-lock-tables -h ${MYSQL_SERVICE_HOST} -P ${MYSQL_SERVICE_PORT:-3306} \
      -u ${MYSQL_USER} --password="$MYSQL_PASSWORD" --all-databases > /var/lib/mysql/data/db_archive_dir/all.sql
    # exit
  2. Remote sync the archive file to your local machine:

    $ oc rsync <existing db container with db archive>:/var/lib/mysql/data/db_archive_dir /tmp/.
  3. Start a second MySQL pod into which to load the database archive file created above. The MySQL pod must have a unique DATABASE_SERVICE_NAME.

    $ oc new-app mysql-persistent \
      -p MYSQL_USER=<archived mysql username> \
      -p MYSQL_PASSWORD=<archived mysql password> \
      -p MYSQL_DATABASE=<archived database name> \
      -p DATABASE_SERVICE_NAME='mysql2' 1
    $ oc rsync /tmp/db_archive_dir new_dbpod1234:/var/lib/mysql/data
    $ oc rsh new_dbpod1234
    1
    mysql is the default. In this example, mysql2 is created.
  4. Use the appropriate commands to restore the database in the new database container from the copied database archive directory:

    MySQL

    $ cd /var/lib/mysql/data/db_archive_dir
    $ mysql -u root
    $ source all.sql
    $ GRANT ALL PRIVILEGES ON <dbname>.* TO '<your username>'@'localhost'; FLUSH PRIVILEGES;
    $ cd ../; rm -rf /var/lib/mysql/data/db_backup_dir

    You now have two MySQL database pods running in your project with the archived database.

25.4. Requirements

The oc rsync command uses the local rsync command if present on the client’s machine. This requires that the remote container also have the rsync command.

If rsync is not found locally or in the remote container, then a tar archive will be created locally and sent to the container where tar will be used to extract the files. If tar is not available in the remote container, then the copy will fail.

The tar copy method does not provide the same functionality as rsync. For example, rsync creates the destination directory if it does not exist and will only send files that are different between the source and the destination.

Note

In Windows, the cwRsync client should be installed and added to the PATH for use with the oc rsync command.

25.5. Specifying the Copy Source

The source argument of the oc rsync command must point to either a local directory or a pod directory. Individual files are not currently supported.

When specifying a pod directory the directory name must be prefixed with the pod name:

<pod name>:<dir>

Just as with standard rsync, if the directory name ends in a path separator (/), only the contents of the directory are copied to the destination. Otherwise, the directory itself is copied to the destination with all its contents.

25.6. Specifying the Copy Destination

The destination argument of the oc rsync command must point to a directory. If the directory does not exist, but rsync is used for copy, the directory is created for you.

25.7. Deleting Files at the Destination

The --delete flag may be used to delete any files in the remote directory that are not in the local directory.

25.8. Continuous Syncing on File Change

Using the --watch option causes the command to monitor the source path for any file system changes, and synchronizes changes when they occur. With this argument, the command runs forever.

Synchronization occurs after short quiet periods to ensure a rapidly changing file system does not result in continuous synchronization calls.

When using the --watch option, the behavior is effectively the same as manually invoking oc rsync repeatedly, including any arguments normally passed to oc rsync. Therefore, you can control the behavior via the same flags used with manual invocations of oc rsync, such as --delete.

25.9. Advanced Rsync Features

The oc rsync command exposes fewer command line options than standard rsync. In the case that you wish to use a standard rsync command line option which is not available in oc rsync (for example the --exclude-from=FILE option), it may be possible to use standard rsync 's --rsh (-e) option or RSYNC_RSH environment variable as a workaround, as follows:

$ rsync --rsh='oc rsh' --exclude-from=FILE SRC POD:DEST

or:

$ export RSYNC_RSH='oc rsh'
$ rsync --exclude-from=FILE SRC POD:DEST

Both of the above examples configure standard rsync to use oc rsh as its remote shell program to enable it to connect to the remote pod, and are an alternative to running oc rsync.

Chapter 26. Port Forwarding

26.1. Overview

OpenShift Online takes advantage of a feature built-in to Kubernetes to support port forwarding to pods. See Architecture for more information.

You can use the CLI to forward one or more local ports to a pod. This allows you to listen on a given or random port locally, and have data forwarded to and from given ports in the pod.

26.2. Basic Usage

Support for port forwarding is built into the CLI:

$ oc port-forward <pod> [<local_port>:]<remote_port> [...[<local_port_n>:]<remote_port_n>]

The CLI listens on each local port specified by the user, forwarding via the protocol described below.

Ports may be specified using the following formats:

5000

The client listens on port 5000 locally and forwards to 5000 in the pod.

6000:5000

The client listens on port 6000 locally and forwards to 5000 in the pod.

:5000 or 0:5000

The client selects a free local port and forwards to 5000 in the pod.

For example, to listen on ports 5000 and 6000 locally and forward data to and from ports 5000 and 6000 in the pod, run:

$ oc port-forward <pod> 5000 6000

To listen on port 8888 locally and forward to 5000 in the pod, run:

$ oc port-forward <pod> 8888:5000

To listen on a free port locally and forward to 5000 in the pod, run:

$ oc port-forward <pod> :5000

Or, alternatively:

$ oc port-forward <pod> 0:5000

26.3. Protocol

Clients initiate port forwarding to a pod by issuing a request to the Kubernetes API server:

/proxy/minions/<node_name>/portForward/<namespace>/<pod>

In the above URL:

  • <node_name> is the FQDN of the node.
  • <namespace> is the namespace of the target pod.
  • <pod> is the name of the target pod.

For example:

/proxy/minions/node123.openshift.com/portForward/myns/mypod

After sending a port forward request to the API server, the client upgrades the connection to one that supports multiplexed streams; the current implementation uses SPDY.

The client creates a stream with the port header containing the target port in the pod. All data written to the stream is delivered via the Kubelet to the target pod and port. Similarly, all data sent from the pod for that forwarded connection is delivered back to the same stream in the client.

The client closes all streams, the upgraded connection, and the underlying connection when it is finished with the port forwarding request.

Note

Administrators can see the Architecture guide for more information.

Chapter 27. Shared Memory

27.1. Overview

There are two types of shared memory objects in Linux: System V and POSIX. The containers in a pod share the IPC namespace of the pod infrastructure container and so are able to share the System V shared memory objects. This document describes how they can also share POSIX shared memory objects.

27.2. POSIX Shared Memory

POSIX shared memory requires that a tmpfs be mounted at /dev/shm. The containers in a pod do not share their mount namespaces so we use volumes to provide the same /dev/shm into each container in a pod. The following example shows how to set up POSIX shared memory between two containers.

shared-memory.yaml

---
apiVersion: v1
id: hello-openshift
kind: Pod
metadata:
  name: hello-openshift
  labels:
    name: hello-openshift
spec:
  volumes:                          1
    - name: dshm
      emptyDir:
        medium: Memory
  containers:
    - image: kubernetes/pause
      name: hello-container1
      ports:
        - containerPort: 8080
          hostPort: 6061
      volumeMounts:                 2
        - mountPath: /dev/shm
          name: dshm
    - image: kubernetes/pause
      name: hello-container2
      ports:
        - containerPort: 8081
          hostPort: 6062
      volumeMounts:                 3
        - mountPath: /dev/shm
          name: dshm

1
specifies the tmpfs volume dshm.
2
enables POSIX shared memory for hello-container1 via dshm.
3
enables POSIX shared memory for hello-container2 via dshm.

Create the pod using the shared-memory.yaml file:

$ oc create -f shared-memory.yaml

Chapter 28. Application Health

28.1. Overview

In software systems, components can become unhealthy due to transient issues (such as temporary connectivity loss), configuration errors, or problems with external dependencies. OpenShift Online applications have a number of options to detect and handle unhealthy containers.

28.2. Container Health Checks Using Probes

A probe is a Kubernetes action that periodically performs diagnostics on a running container. Currently, two types of probes exist, each serving a different purpose:

Liveness Probe

A liveness probe checks if the container in which it is configured is still running. If the liveness probe fails, the kubelet kills the container, which will be subjected to its restart policy. Set a liveness check by configuring the template.spec.containers.livenessprobe stanza of a pod configuration.

Readiness Probe

A readiness probe determines if a container is ready to service requests. If the readiness probe fails a container, the endpoints controller ensures the container has its IP address removed from the endpoints of all services. A readiness probe can be used to signal to the endpoints controller that even though a container is running, it should not receive any traffic from a proxy. Set a readiness check by configuring the template.spec.containers.readinessprobe stanza of a pod configuration.

The exact timing of a probe is controlled by two fields, both expressed in units of seconds:

FieldDescription

initialDelaySeconds

How long to wait after the container starts to begin the probe.

timeoutSeconds

How long to wait for the probe to finish (default: 1). If this time is exceeded, OpenShift Online considers the probe to have failed.

Both probes can be configured in three ways:

HTTP Checks

The kubelet uses a web hook to determine the healthiness of the container. The check is deemed successful if the HTTP response code is between 200 and 399. The following is an example of a readiness check using the HTTP checks method:

Example 28.1. Readiness HTTP check

...
readinessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

A HTTP check is ideal for applications that return HTTP status codes when completely initialized.

Container Execution Checks

The kubelet executes a command inside the container. Exiting the check with status 0 is considered a success. The following is an example of a liveness check using the container execution method:

Example 28.2. Liveness Container Execution Check

...
livenessProbe:
  exec:
    command:
    - cat
    - /tmp/health
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

TCP Socket Checks

The kubelet attempts to open a socket to the container. The container is only considered healthy if the check can establish a connection. The following is an example of a liveness check using the TCP socket check method:

Example 28.3. Liveness TCP Socket Check

...
livenessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 15
  timeoutSeconds: 1
...

A TCP socket check is ideal for applications that do not start listening until initialization is complete.

For more information on health checks, see the Kubernetes documentation.

Chapter 29. Events

29.1. Overview

Events in OpenShift Online are modeled based on events that happen to API objects in an OpenShift Online cluster. Events allow OpenShift Online to record information about real-world events in a resource-agnostic manner. They also allow developers and administrators to consume information about system components in a unified way.

29.2. Viewing Events with the CLI

You can get a list of events in a given project using the following command:

$ oc get events [-n <project>]

29.3. Viewing Events in the Console

You can see events in your project from the web console from the BrowseEvents page. Many other objects, such as pods and deployments, have their own Events tab as well, which shows events related to that object.

29.4. Failure Notifications

For each of your projects, you can choose to receive email notifications about various failures, including dead or failed deployments, dead builds, and dead or failed persistent volume claims (PVCs).

29.5. Comprehensive List of Events

This section describes the events of OpenShift Online.

Table 29.1. Configuration Events

NameDescription

FailedValidation

Failed pod configuration validation.

Table 29.2. Container Events

NameDescription

BackOff

Back-off restarting failed the container.

Created

Container created.

Failed

Pull/Create/Start failed.

Killing

Killing the container.

Started

Container started.

Table 29.3. Health Events

NameDescription

Unhealthy

Container is unhealthy.

Table 29.4. Image Events

NameDescription

BackOff

Back off Ctr Start, image pull.

ErrImageNeverPull

The image’s NeverPull Policy is violated.

Failed

Failed to pull the image.

InspectFailed

Failed to inspect the image.

Pulled

Successfully pulled the image or the container image is already present on the machine.

Pulling

Pulling the image.

Table 29.5. Image Manager Events

NameDescription

FreeDiskSpaceFailed

Free disk space failed.

InvalidDiskCapacity

Invalid disk capacity.

Table 29.6. Node Events

NameDescription

FailedMount

Volume mount failed.

HostNetworkNotSupported

Host network not supported.

HostPortConflict

Host/port conflict.

InsufficientFreeCPU

Insufficient free CPU.

InsufficientFreeMemory

Insufficient free memory.

KubeletSetupFailed

Kubelet setup failed.

NilShaper

Undefined shaper.

NodeNotReady

Node is not ready.

NodeNotSchedulable

Node is not schedulable.

NodeReady

Node is ready.

NodeSchedulable

Node is schedulable.

NodeSelectorMismatching

Node selector mismatch.

OutOfDisk

Out of disk.

Rebooted

Node rebooted.

Starting

Starting kubelet.

Table 29.7. Pod Worker Events

NameDescription

FailedSync

Pod sync failed.

Table 29.8. System Events

NameDescription

SystemOOM

There is an OOM (out of memory) situation on the cluster.

Chapter 30. Notifications

30.1. Overview

For each of your projects, you can choose to receive email notifications about various failures, including:

  • Dead deployment: A failed deployment in which no other deployment in the rollout history is able to remain active. The related deployment configuration has no deployments with active replicas.
  • Failed deployment: A new deployment fails and another deployment in the rollout history is now active. The related deployment configuration has one active deployment, but it is not the latest deployment.
  • Stuck deployment: A deployment in which the active replicas are less than the replicas defined by the deployment configuration for a significant amount of time.
  • Dead build: Occurs when automatically triggered builds are failing for a specified build configuration that previously had successful builds.
  • Dead persistent volume claim: A claim that is in the pending state for longer than some reasonable amount of time. These are claims whose resource requests likely do not match with any existing persistent volumes in the cluster.
  • Lost persistent volume claim: A claim that is bound to a backing persistent volume, but the persistent volume has unexpectedly gone away.

30.2. How It Works

The Notifications feature continually monitors resources across the cluster. When problems are detected, notifications are sent to the email address of the project creator, designated in the requester field of the project. These notifications are throttled so that the recipient is not overwhelmed with email messages.

30.3. Configuring Notifications via the Web Console

From the web console:

  1. Select Monitoring from the left-hand navigation.
  2. Select Notifications from the Monitoring dropdown menu.

    Monitoring Menu
  3. Select your preferences on the Notifications page.

    Notifications Menu
  4. Click the Save button at the bottom of the page.

    Once your options are successfully saved, this message will appear at the top of the page:

    Success Message

30.4. Storing Preferences

The Notifications feature uses a ConfigMap named openshift-online-notifications in each namespace to store preferences. The preferences allow users to specify which resources they would like to receive notifications for. An interface is provided in the web console to easily modify these preferences.

Example ConfigMap for Notification Preferences

[source,yaml]
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: openshift-online-notifications
  namespace: example
data:
  builds-enabled: 'true'
  deployments-enabled: 'true'
  storage-enabled: 'true'

If the ConfigMap is not present, it is assumed that notifications are disabled. The email address used is the email address of the project owner.

Chapter 31. Downward API

31.1. Overview

The downward API is a mechanism that allows containers to consume information about API objects without coupling to OpenShift Online. Such information includes the pod’s name, namespace, and resource values. Containers can consume information from the downward API using environment variables or a volume plug-in.

31.2. Selecting Fields

Fields within the pod are selected using the FieldRef API type. FieldRef has two fields:

FieldDescription

fieldPath

The path of the field to select, relative to the pod.

apiVersion

The API version to interpret the fieldPath selector within.

Currently, the valid selectors in the v1 API include:

SelectorDescription

metadata.name

The pod’s name. This is supported in both environment variables and volumes.

metadata.namespace

The pod’s namespace.This is supported in both environment variables and volumes.

metadata.labels

The pod’s labels. This is only supported in volumes and not in environment variables.

metadata.annotations

The pod’s annotations. This is only supported in volumes and not in environment variables.

status.podIP

The pod’s IP. This is only supported in environment variables and not volumes.

The apiVersion field, if not specified, defaults to the API version of the enclosing pod template.

31.3. Consuming Container Values Using the Downward API

31.3.1. Using Environment Variables

One mechanism for consuming the downward API is using a container’s environment variables. The EnvVar type’s valueFrom field (of type EnvVarSource) is used to specify that the variable’s value should come from a FieldRef source instead of the literal value specified by the value field. In the future, additional sources may be supported; currently the source’s fieldRef field is used to select a field from the downward API.

Only constant attributes of the pod can be consumed this way, as environment variables cannot be updated once a process is started in a way that allows the process to be notified that the value of a variable has changed. The fields supported using environment variables are:

  • Pod name
  • Pod namespace

    1. Create a pod.yaml file:

      apiVersion: v1
      kind: Pod
      metadata:
        name: dapi-env-test-pod
      spec:
        containers:
          - name: env-test-container
            image: gcr.io/google_containers/busybox
            command: [ "/bin/sh", "-c", "env" ]
            env:
              - name: MY_POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: MY_POD_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
        restartPolicy: Never
    2. Create the pod from the pod.yaml file:

      $ oc create -f pod.yaml
    3. Check the container’s logs for the MY_POD_NAME and MY_POD_NAMESPACE values:

      $ oc logs -p dapi-env-test-pod

31.3.2. Using the Volume Plug-in

Another mechanism for consuming the downward API is using a volume plug-in. The downward API volume plug-in creates a volume with configured fields projected into files. The metadata field of the VolumeSource API object is used to configure this volume. The plug-in supports the following fields:

  • Pod name
  • Pod namespace
  • Pod annotations
  • Pod labels

Example 31.1. Downward API Volume Plug-in Configuration

spec:
  volumes:
    - name: podinfo
      metadata: 1
        items:  2
          - name: "labels" 3
            fieldRef:
              fieldPath: metadata.labels 4
1
The metadata field of the volume source configures the downward API volume.
2
The items field holds a list of fields to project into the volume.
3
The name of the file to project the field into.
4
The selector of the field to project.

For example:

  1. Create a volume-pod.yaml file:

    kind: Pod
    apiVersion: v1
    metadata:
      labels:
        zone: us-east-coast
        cluster: downward-api-test-cluster1
        rack: rack-123
      name: dapi-volume-test-pod
      annotations:
        annotation1: "345"
        annotation2: "456"
    spec:
      containers:
        - name: volume-test-container
          image: gcr.io/google_containers/busybox
          command: ["sh", "-c", "cat /etc/labels /etc/annotations"]
          volumeMounts:
            - name: podinfo
              mountPath: /etc
              readOnly: false
      volumes:
        - name: podinfo
          metadata:
            items:
              - name: "labels"
                fieldRef:
                  fieldPath: metadata.labels
              - name: "annotations"
                fieldRef:
                  fieldPath: metadata.annotations
      restartPolicy: Never
  2. Create the pod from the volume-pod.yaml file:

    $ oc create -f volume-pod.yaml
  3. Check the container’s logs and verify the presence of the configured fields:

    $ oc logs -p dapi-volume-test-pod
    cluster=downward-api-test-cluster1
    rack=rack-123
    zone=us-east-coast
    annotation1=345
    annotation2=456
    kubernetes.io/config.source=api

31.4. Consuming Container Resources Using the Downward API

When creating pods, you can use the downward API to inject information about computing resource requests and limits so that image and application authors can correctly create an image for specific environments.

You can do this using both the environment variable and volume plug-in methods.

31.4.1. Using Environment Variables

  1. When creating a pod configuration, specify environment variables that correspond to the contents of the resources field in the spec.container field:

    ....
    spec:
      containers:
        - name: test-container
          image: gcr.io/google_containers/busybox:1.24
          command: [ "/bin/sh", "-c", "env" ]
          resources:
            requests:
              memory: "32Mi"
              cpu: "125m"
            limits:
              memory: "64Mi"
              cpu: "250m"
          env:
            - name: MY_CPU_REQUEST
              valueFrom:
                resourceFieldRef:
                  resource: requests.cpu
            - name: MY_CPU_LIMIT
              valueFrom:
                resourceFieldRef:
                  resource: limits.cpu
            - name: MY_MEM_REQUEST
              valueFrom:
                resourceFieldRef:
                  resource: requests.memory
            - name: MY_MEM_LIMIT
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
    ....

    If the resource limits are not included in the container configuration, the downward API defaults to the node’s CPU and memory allocatable values.

  2. Create the pod from the pod.yaml file:

    $ oc create -f pod.yaml

31.4.2. Using the Volume Plug-in

  1. When creating a pod configuration, use the spec.volumes.downwardAPI.items field to describe the desired resources that correspond to the spec.resources field:

    ....
    spec:
      containers:
        - name: client-container
          image: gcr.io/google_containers/busybox:1.24
          command: ["sh", "-c", "while true; do echo; if [[ -e /etc/cpu_limit ]]; then cat /etc/cpu_limit; fi; if [[ -e /etc/cpu_request ]]; then cat /etc/cpu_request; fi; if [[ -e /etc/mem_limit ]]; then cat /etc/mem_limit; fi; if [[ -e /etc/mem_request ]]; then cat /etc/mem_request; fi; sleep 5; done"]
          resources:
            requests:
              memory: "32Mi"
              cpu: "125m"
            limits:
              memory: "64Mi"
              cpu: "250m"
          volumeMounts:
            - name: podinfo
              mountPath: /etc
              readOnly: false
      volumes:
        - name: podinfo
          downwardAPI:
            items:
              - path: "cpu_limit"
                resourceFieldRef:
                  containerName: client-container
                  resource: limits.cpu
              - path: "cpu_request"
                resourceFieldRef:
                  containerName: client-container
                  resource: requests.cpu
              - path: "mem_limit"
                resourceFieldRef:
                  containerName: client-container
                  resource: limits.memory
              - path: "mem_request"
                resourceFieldRef:
                  containerName: client-container
                  resource: requests.memory
    ....

    If the resource limits are not included in the container configuration, the downward API defaults to the node’s CPU and memory allocatable values.

  2. Create the pod from the volume-pod.yaml file:

    $ oc create -f volume-pod.yaml

31.5. Consuming Secrets Using the Downward API

When creating pods, you can use the downward API to inject Secrets so image and application authors can create an image for specific environments.

31.5.1. Using Environment Variables

  1. Create a secret.yaml file:

    apiVersion: v1
    kind: Secret
    metadata:
      name: mysecret
    data:
      password: cGFzc3dvcmQ=
      username: ZGV2ZWxvcGVy
    type: kubernetes.io/basic-auth
  2. Create a Secret from the secret.yaml file:

    oc create -f secret.yaml
  3. Create a pod.yaml file that references the username field from the above Secret:

    apiVersion: v1
    kind: Pod
    metadata:
      name: dapi-env-test-pod
    spec:
      containers:
        - name: env-test-container
          image: gcr.io/google_containers/busybox
          command: [ "/bin/sh", "-c", "env" ]
          env:
            - name: MY_SECRET_USERNAME
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: username
      restartPolicy: Never
  4. Create the pod from the pod.yaml file:

    $ oc create -f pod.yaml
  5. Check the container’s logs for the MY_SECRET_USERNAME value:

    $ oc logs -p dapi-env-test-pod

31.6. Consuming ConfigMaps Using the Downward API

When creating pods, you can use the downward API to inject ConfigMap values so image and application authors can create an image for specific environments.

31.6.1. Using Environment Variables

  1. Create a configmap.yaml file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: myconfigmap
    data:
      mykey: myvalue
  2. Create a ConfigMap from the configmap.yaml file:

    oc create -f configmap.yaml
  3. Create a pod.yaml file that references the above ConfigMap:

    apiVersion: v1
    kind: Pod
    metadata:
      name: dapi-env-test-pod
    spec:
      containers:
        - name: env-test-container
          image: gcr.io/google_containers/busybox
          command: [ "/bin/sh", "-c", "env" ]
          env:
            - name: MY_CONFIGMAP_VALUE
              valueFrom:
                configMapKeyRef:
                  name: myconfigmap
                  key: mykey
      restartPolicy: Never
  4. Create the pod from the pod.yaml file:

    $ oc create -f pod.yaml
  5. Check the container’s logs for the MY_CONFIGMAP_VALUE value:

    $ oc logs -p dapi-env-test-pod

31.7. Environment Variable References

When creating pods, you can reference the value of a previously defined environment variable by using the $() syntax. If the environment variable reference can not be resolved, the value will be left as the provided string.

31.7.1. Using Environment Variable References

  1. Create a pod.yaml file that references an existing environment variable:

    apiVersion: v1
    kind: Pod
    metadata:
      name: dapi-env-test-pod
    spec:
      containers:
        - name: env-test-container
          image: gcr.io/google_containers/busybox
          command: [ "/bin/sh", "-c", "env" ]
          env:
            - name: MY_EXISTING_ENV
              value: my_value
            - name: MY_ENV_VAR_REF_ENV
              value: $(MY_EXISTING_ENV)
      restartPolicy: Never
  2. Create the pod from the pod.yaml file:

    $ oc create -f pod.yaml
  3. Check the container’s logs for the MY_ENV_VAR_REF_ENV value:

    $ oc logs -p dapi-env-test-pod

31.7.2. Escaping Environment Variable References

When creating a pod, you can escape an environment variable reference by using a double dollar sign. The value will then be set to a single dollar sign version of the provided value.

  1. Create a pod.yaml file that references an existing environment variable:

    apiVersion: v1
    kind: Pod
    metadata:
      name: dapi-env-test-pod
    spec:
      containers:
        - name: env-test-container
          image: gcr.io/google_containers/busybox
          command: [ "/bin/sh", "-c", "env" ]
          env:
            - name: MY_NEW_ENV
              value: $$(SOME_OTHER_ENV)
      restartPolicy: Never
  2. Create the pod from the pod.yaml file:

    $ oc create -f pod.yaml
  3. Check the container’s logs for the MY_NEW_ENV value:

    $ oc logs -p dapi-env-test-pod

Chapter 32. Managing Environment Variables

32.1. Setting and Unsetting Environment Variables

OpenShift Online provides the oc set env command to set or unset environment variables for objects that have a pod template, such as replication controllers or deployment configurations. It can also list environment variables in pods or any object that has a pod template. This command can also be used on BuildConfig objects.

32.2. List Environment Variables

To list environment variables in pods or pod templates:

$ oc set env <object-selection> --list [<common-options>]

This example lists all environment variables for pod p1:

$ oc set env pod/p1 --list

32.3. Set Environment Variables

To set environment variables in the pod templates:

$ oc set env <object-selection> KEY_1=VAL_1 ... KEY_N=VAL_N [<set-env-options>] [<common-options>]

Set environment options:

OptionDescription

-e, --env=<KEY>=<VAL>

Set given key value pairs of environment variables.

--overwrite

Confirm updating existing environment variables.

In the following example, both commands modify environment variable STORAGE in the deployment config registry. The first adds, with value /data. The second updates, with value /opt.

$ oc set env dc/registry STORAGE=/data
$ oc set env dc/registry --overwrite STORAGE=/opt

The following example finds environment variables in the current shell whose names begin with RAILS_ and adds them to the replication controller r1 on the server:

$ env | grep RAILS_ | oc set env rc/r1 -e -

The following example does not modify the replication controller defined in file rc.json. Instead, it writes a YAML object with updated environment STORAGE=/local to new file rc.yaml.

$ oc set env -f rc.json STORAGE=/opt -o yaml > rc.yaml

32.3.1. Automatically Added Environment Variables

Table 32.1. Automatically Added Environment Variables

Variable Name

<SVCNAME>_SERVICE_HOST

<SVCNAME>_SERVICE_PORT

Example Usage

The service KUBERNETES which exposes TCP port 53 and has been allocated cluster IP address 10.0.0.11 produces the following environment variables:

KUBERNETES_SERVICE_PORT=53
MYSQL_DATABASE=root
KUBERNETES_PORT_53_TCP=tcp://10.0.0.11:53
KUBERNETES_SERVICE_HOST=10.0.0.11
Note

Use the oc rsh command to SSH into your container and run oc set env to list all available variables.

32.4. Unset Environment Variables

To unset environment variables in the pod templates:

$ oc set env <object-selection> KEY_1- ... KEY_N- [<common-options>]
Important

The trailing hyphen (-, U+2D) is required.

This example removes environment variables ENV1 and ENV2 from deployment config d1:

$ oc set env dc/d1 ENV1- ENV2-

This removes environment variable ENV from all replication controllers:

$ oc set env rc --all ENV-

This removes environment variable ENV from container c1 for replication controller r1:

$ oc set env rc r1 --containers='c1' ENV-

Chapter 33. Jobs

33.1. Overview

A job, in contrast to a replication controller, runs a pod with any number of replicas to completion. A job tracks the overall progress of a task and updates its status with information about active, succeeded, and failed pods. Deleting a job will clean up any pod replicas it created. Jobs are part of the Kubernetes API, which can be managed with oc commands like other object types.

See the Kubernetes documentation for more information about jobs.

33.2. Creating a Job

A job configuration consists of the following key parts:

  • A pod template, which describes the application the pod will create.
  • An optional parallelism parameter, which specifies how many pod replicas running in parallel should execute a job. If not specified, this defaults to the value in the completions parameter.
  • An optional completions parameter, specifying how many concurrently running pods should execute a job. If not specified, this value defaults to one.

The following is an example of a job resource:

apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  parallelism: 1    1
  completions: 1    2
  template:         3
    metadata:
      name: pi
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  1. Optional value for how many pod replicas a job should run in parallel; defaults to completions.
  2. Optional value for how many successful pod completions are needed to mark a job completed; defaults to one.
  3. Template for the pod the controller creates.

33.3. Scaling a Job

A job can be scaled up or down by using the oc scale command with the --replicas option, which, in the case of jobs, modifies the spec.parallelism parameter. This will result in modifying the number of pod replicas running in parallel, executing a job.

The following command uses the example job above, and sets the parallelism parameter to three:

$ oc scale job pi --replicas=3
Note

Scaling replication controllers also uses the oc scale command with the --replicas option, but instead changes the replicas parameter of a replication controller configuration.

33.4. Setting Maximum Duration

When defining a Job, you can define its maximum duration by setting the activeDeadlineSeconds field. It is specified in seconds and is not set by default. When not set, there is no maximum duration enforced.

The maximum duration is counted from the time when a first pod gets scheduled in the system, and defines how long a job can be active. It tracks overall time of an execution and is irrelevant to the number of completions (number of pod replicas needed to execute a task). After reaching the specified timeout, the job is terminated by OpenShift Online.

The following example shows the part of a Job specifying activeDeadlineSeconds field for 30 minutes:

  spec:
    activeDeadlineSeconds: 1800

Chapter 34. Cron Jobs

34.1. Overview

A cron job builds on a regular job by allowing you to specifically schedule how the job should be run. Cron jobs are part of the Kubernetes API, which can be managed with oc commands like other object types.

Important

Cron Jobs is a feature currently in Technology Preview.

Cron Jobs are only available for OpenShift Online Pro. For more information about the differences between Starter and Pro tiers, visit the pricing page.

Warning

A cron job creates a job object approximately once per execution time of its schedule, but there are circumstances in which it will fail to create a job or two jobs might be created. As a result, jobs should be idempotent.

34.2. Creating a Cron Job

A cron job configuration consists of the following key parts:

  • A schedule specified in cron format.
  • A job template used when creating the next job.
  • An optional deadline (in seconds) for starting the job if it misses its scheduled time for any reason. Missed jobs executions will be counted as failed ones. If not specified, there is no deadline.
  • ConcurrencyPolicy: An optional concurrency policy, specifying how to treat concurrent jobs within a cron job. Only one of the following concurrent policies may be specified. If not specified, this defaults to allowing concurrent executions.

    • Allow allows Cron Jobs to run concurrently.
    • Forbid forbids concurrent runs, skipping the next run if the previous has not finished yet.
    • Replace cancels the currently running job and replaces it with a new one.
  • An optional flag allowing the suspension of a cron job. If set to true, all subsequent executions will be suspended.

The following is an example of a CronJob resource:

apiVersion: batch/v2alpha1
kind: CronJob
metadata:
  name: pi
spec:
  schedule: "*/1 * * * *"  1
  jobTemplate:             2
    spec:
      template:
        spec:
          containers:
          - name: pi
            image: perl
            command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
          restartPolicy: Never
  1. Schedule for the job. In this example, a job will run every minute.
  2. Job template. This is similar to the job example.
  3. Sets a label for jobs spawned by this cron job.
  4. The restart policy of the pod. This does not apply to the job controller.

You can also create and launch a cron job from a single command using oc run. The following command creates and launches the same cron job as specified in the previous example:

$ oc run pi --image=perl --schedule='*/1 * * * *' \
    --restart=OnFailure --labels parent="cronjobpi" \
    --command -- perl -Mbignum=bpi -wle 'print bpi(2000)'

With oc run, the --schedule option accepts schedules in cron format.

Note

When creating a cron job, oc run only supports the Never or OnFailure restart policies (--restart).

Tip

Delete cron jobs that you no longer need:

$ oc delete cronjob/<cron_job_name>

Doing this prevents them from generating unnecessary artifacts.

34.3. Cleaning Up After a Cron Job

Cron jobs can leave behind artifact resources such as jobs or pods. Check if any remain:

$ oc get jobs
$ oc get pods

All artifacts left over from a job execution use the job name as their prefix. For example, given the cron job example:

$ oc get jobs
NAME            DESIRED   SUCCESSFUL   AGE
pi-1497848100   1         1            1m
pi-1497848160   1         1            49s

$ oc get pods
NAME                  READY     STATUS      RESTARTS   AGE
pi-1497848100-lxs4k   0/1       Completed   0          2m
pi-1497848160-6r0c8   0/1       Completed   0          59s

Delete each artifact if you no longer need them. To delete all jobs spawned by a cron job, specify the label set during cron job creation:

$ oc delete jobs -l <label>

For example, to delete only the jobs generated by the cron job example:

$ oc delete jobs -l parent=cronjobpi
job "pi-1497848100" deleted
job "pi-1497848160" deleted

Chapter 35. Create from URL

35.1. Overview

Create From URL is a function that allows you to construct a URL from an image stream, image tag, or template.

Create from URL only works with image streams or templates from namespaces that have been explicitly whitelisted. The whitelist contains the openshift namespace by default.

You can define custom buttons.

Run on OpenShift Buttons

These buttons leverage a defined URL pattern with an appropriate query string. The user is prompted to select the project. Then, the Create from URL workflow continues.

35.2. Using an Image Stream and Image Tag

35.2.1. Query String Parameters

NameDescriptionRequiredSchemaDefault

imageStream

The value metadata.name as defined in the image stream to be used.

true

string

 

imageTag

The value spec.tags.name as defined in the image stream to be used.

true

string

 

namespace

The name of the namespace containing the image stream and image tag to use.

false

string

openshift

name

Identifies the resources created for this application.

false

string

 

sourceURI

The Git repository URL containing the application source code.

false

string

 

sourceRef

The branch, tag, or commit for the application source code specified in sourceURI.

false

string

 

contextDir

The subdirectory for the application source code specified in sourceURI, used as the context directory for the build.

false

string

 
Note

Reserved characters in parameter values should be URL encoded.

35.2.1.1. Example

 create?imageStream=nodejs&imageTag=4&name=nodejs&sourceURI=https%3A%2F%2Fgithub.com%2Fopenshift%2Fnodejs-ex.git&sourceRef=master&contextDir=%2F

35.3. Using a Template

35.3.1. Query String Parameters

NameDescriptionRequiredSchemaDefault

template

The value of metadata.name as defined in the template to be used.

true

string

 

templateParamsMap

A JSON parameters map containing the template parameter name and corresponding value you wish to override.

false

JSON

 

namespace

The name of the namespace containing the template to use.

false

string

openshift

Note

Reserved characters in parameter values should be URL encoded.

35.3.1.1. Example

 create?template=nodejs-mongodb-example&templateParamsMap={"SOURCE_REPOSITORY_URL"%3A"https%3A%2F%2Fgithub.com%2Fopenshift%2Fnodejs-ex.git"}

Legal Notice

Copyright © 2017 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.