Red Hat Training

A Red Hat training course is available for OpenShift Container Platform

Chapter 2. Application Life Cycle Management

2.1. Planning Your Development Process

2.1.1. Overview

OpenShift Container Platform is designed for building and deploying applications. Depending on how much you want to involve OpenShift Container Platform in your development process, you can choose to:

  • focus your development within an OpenShift Container Platform project, using it to build an application from scratch then continuously develop and manage its lifecycle, or
  • bring an application (e.g., binary, container image, source code) you have already developed in a separate environment and deploy it onto OpenShift Container Platform.

2.1.2. Using OpenShift Container Platform as Your Development Environment

Deploying then developing on OpenShift Container Platform

You can begin your application’s development from scratch using OpenShift Container Platform directly. Consider the following steps when planning this type of development process:

Initial Planning

  • What does your application do?
  • What programming language will it be developed in?

Access to OpenShift Container Platform

  • OpenShift Container Platform should be installed by this point, either by yourself or an administrator within your organization.

Develop

  • Using your editor or IDE of choice, create a basic skeleton of an application. It should be developed enough to tell OpenShift Container Platform what kind of application it is.
  • Push the code to your Git repository.

Generate

  • Create a basic application using the oc new-app command. OpenShift Container Platform generates build and deployment configurations.

Manage

  • Start developing your application code.
  • Ensure your application builds successfully.
  • Continue to locally develop and polish your code.
  • Push your code to a Git repository.
  • Is any extra configuration needed? Explore the Developer Guide for more options.

Verify

  • You can verify your application in a number of ways. You can push your changes to your application’s Git repository, and use OpenShift Container Platform to rebuild and redeploy your application. Alternatively, you can hot deploy using rsync to synchronize your code changes into a running pod.

2.1.3. Bringing an Application to Deploy on OpenShift Container Platform

Developing then deploying on OpenShift Container Platform

Another possible application development strategy is to develop locally, then use OpenShift Container Platform to deploy your fully developed application. Use the following process if you plan to have application code already, then want to build and deploy onto an OpenShift Container Platform installation when completed:

Initial Planning

  • What does your application do?
  • What programming language will it be developed in?

Develop

  • Develop your application code using your editor or IDE of choice.
  • Build and test your application code locally.
  • Push your code to a Git repository.

Access to OpenShift Container Platform

  • OpenShift Container Platform should be installed by this point, either by yourself or an administrator within your organization.

Generate

  • Create a basic application using the oc new-app command. OpenShift Container Platform generates build and deployment configurations.

Verify

  • Ensure that the application that you have built and deployed in the above Generate step is successfully running on OpenShift Container Platform.

Manage

  • Continue to develop your application code until you are happy with the results.
  • Rebuild your application in OpenShift Container Platform to accept any newly pushed code.
  • Is any extra configuration needed? Explore the Developer Guide for more options.

2.2. Creating New Applications

2.2.1. Overview

You can create a new OpenShift Container Platform application from components including source or binary code, images and/or templates by using either the OpenShift CLI or web console.

2.2.2. Creating an Application Using the CLI

2.2.2.1. Creating an Application From Source Code

The new-app command allows you to create applications from source code in a local or remote Git repository.

To create an application using a Git repository in a local directory:

$ oc new-app /path/to/source/code
Note

If using a local Git repository, the repository should have a remote named origin that points to a URL accessible by the OpenShift Container Platform cluster. If there is no recognised remote, new-app will create a binary build.

To create an application using a remote Git repository:

$ oc new-app https://github.com/sclorg/cakephp-ex

To create an application using a private remote Git repository:

$ oc new-app https://github.com/youruser/yourprivaterepo --source-secret=yoursecret
Note

If using a private remote Git repository, you can use the --source-secret flag to specify an existing source clone secret that will get injected into your BuildConfig to access the repository.

You can use a subdirectory of your source code repository by specifying a --context-dir flag. To create an application using a remote Git repository and a context subdirectory:

$ oc new-app https://github.com/sclorg/s2i-ruby-container.git \
    --context-dir=2.0/test/puma-test-app

Also, when specifying a remote URL, you can specify a Git branch to use by appending #<branch_name> to the end of the URL:

$ oc new-app https://github.com/openshift/ruby-hello-world.git#beta4

The new-app command creates a build configuration, which itself creates a new application image from your source code. The new-app command typically also creates a deployment configuration to deploy the new image, and a service to provide load-balanced access to the deployment running your image.

OpenShift Container Platform automatically detects whether the Docker, Pipeline or Sourcebuild strategy should be used, and in the case of Source builds, detects an appropriate language builder image.

Build Strategy Detection

If a Jenkinsfile exists in the root or specified context directory of the source repository when creating a new application, OpenShift Container Platform generates a Pipeline build strategy. Otherwise, if a Dockerfile is found, OpenShift Container Platform generates a Docker build strategy. Otherwise, it generates a Source build strategy.

You can override the build strategy by setting the --strategy flag to either docker, pipeline or source.

$ oc new-app /home/user/code/myapp --strategy=docker
Note

The oc command requires that files containing build sources are available in a remote Git repository. For all source builds, you must use git remote -v.

Language Detection

If using the Source build strategy, new-app attempts to determine the language builder to use by the presence of certain files in the root or specified context directory of the repository:

Table 2.1. Languages Detected by new-app

LanguageFiles

dotnet

project.json, *.csproj

jee

pom.xml

nodejs

app.json, package.json

perl

cpanfile, index.pl

php

composer.json, index.php

python

requirements.txt, setup.py

ruby

Gemfile, Rakefile, config.ru

scala

build.sbt

golang

Godeps, main.go

After a language is detected, new-app searches the OpenShift Container Platform server for image stream tags that have a supports annotation matching the detected language, or an image stream that matches the name of the detected language. If a match is not found, new-app searches the Docker Hub registry for an image that matches the detected language based on name.

You can override the image the builder uses for a particular source repository by specifying the image (either an image stream or container specification) and the repository, with a ~ as a separator. Note that if this is done, build strategy detection and language detection are not carried out.

For example, to use the myproject/my-ruby image stream with the source in a remote repository:

$ oc new-app myproject/my-ruby~https://github.com/openshift/ruby-hello-world.git

To use the openshift/ruby-20-centos7:latest container image stream with the source in a local repository:

$ oc new-app openshift/ruby-20-centos7:latest~/home/user/code/my-ruby-app
Note

Language detection requires the Git client to be locally installed so that your repository can be cloned and inspected. If Git is not available, you can avoid the language detection step by specifying the builder image to use with your repository with the <image>~<repository> syntax.

Using the -i <image> <repository> invocation requires that new-app attempt to clone repository in order to determine what type of artifact it is, so this will fail if Git is not available.

Similarly using the -i <image> --code <repository> invocation requires new-app clone repository in order to determine whether image should be used as a builder for the source code, or deployed separately, as in the case of a database image.

2.2.2.2. Creating an Application From an Image

You can deploy an application from an existing image. Images can come from image streams in the OpenShift Container Platform server, images in a specific registry or Docker Hub registry, or images in the local Docker server.

The new-app command attempts to determine the type of image specified in the arguments passed to it. However, you can explicitly tell new-app whether the image is a container image (using the --docker-image argument) or an image stream (using the -i|--image argument).

Note

If you specify an image from your local Docker repository, you must ensure that the same image is available to the OpenShift Container Platform cluster nodes.

For example, to create an application from the DockerHub MySQL image:

$ oc new-app mysql

To create an application using an image in a private registry, specify the full container image specification:

$ oc new-app myregistry:5000/example/myimage
Note

If the registry containing the image is not secured with SSL, cluster administrators must ensure that the Docker daemon on the OpenShift Container Platform node hosts is run with the --insecure-registry flag pointing to that registry. You must also tell new-app that the image comes from an insecure registry with the --insecure-registry flag.

You can create an application from an existing image stream and optional image stream tag:

$ oc new-app my-stream:v1

2.2.2.3. Creating an Application From a Template

You can create an application from a previously stored template or from a template file, by specifying the name of the template as an argument. For example, you can store a sample application template and use it to create an application.

To create an application from a stored template:

$ oc create -f examples/sample-app/application-template-stibuild.json
$ oc new-app ruby-helloworld-sample

To directly use a template in your local file system, without first storing it in OpenShift Container Platform, use the -f|--file argument:

$ oc new-app -f examples/sample-app/application-template-stibuild.json

Template Parameters

When creating an application based on a template, use the -p|--param argument to set parameter values defined by the template:

$ oc new-app ruby-helloworld-sample \
    -p ADMIN_USERNAME=admin -p ADMIN_PASSWORD=mypassword

You can store your parameters in a file, then use that file with --param-file when instantiating a template. If you want to read the parameters from standard input, use --param-file=-:

$ cat helloworld.params
ADMIN_USERNAME=admin
ADMIN_PASSWORD=mypassword
$ oc new-app ruby-helloworld-sample --param-file=helloworld.params
$ cat helloworld.params | oc new-app ruby-helloworld-sample --param-file=-

2.2.2.4. Further Modifying Application Creation

The new-app command generates OpenShift Container Platform objects that will build, deploy, and run the application being created. Normally, these objects are created in the current project using names derived from the input source repositories or the input images. However, new-app allows you to modify this behavior.

The set of objects created by new-app depends on the artifacts passed as input: source repositories, images, or templates.

Table 2.2. new-app Output Objects

ObjectDescription

BuildConfig

A BuildConfig is created for each source repository specified in the command line. The BuildConfig specifies the strategy to use, the source location, and the build output location.

ImageStreams

For BuildConfig, two ImageStreams are usually created. One represents the input image. With Source builds, this is the builder image. With Docker builds, this is the FROM image. The second one represents the output image. If a container image was specified as input to new-app, then an image stream is created for that image as well.

DeploymentConfig

A DeploymentConfig is created either to deploy the output of a build, or a specified image. The new-app command creates emptyDir volumes for all Docker volumes that are specified in containers included in the resulting DeploymentConfig.

Service

The new-app command attempts to detect exposed ports in input images. It uses the lowest numeric exposed port to generate a service that exposes that port. In order to expose a different port, after new-app has completed, simply use the oc expose command to generate additional services.

Other

Other objects may be generated when instantiating templates, according to the template.

2.2.2.4.1. Specifying Environment Variables

When generating applications from a template, source, or an image, you can use the -e|--env argument to pass environment variables to the application container at run time:

$ oc new-app openshift/postgresql-92-centos7 \
    -e POSTGRESQL_USER=user \
    -e POSTGRESQL_DATABASE=db \
    -e POSTGRESQL_PASSWORD=password

The variables can also be read from file using the --env-file argument:

$ cat postgresql.env
POSTGRESQL_USER=user
POSTGRESQL_DATABASE=db
POSTGRESQL_PASSWORD=password
$ oc new-app openshift/postgresql-92-centos7 --env-file=postgresql.env

Additionally, environment variables can be given on standard input by using --env-file=-:

$ cat postgresql.env | oc new-app openshift/postgresql-92-centos7 --env-file=-

See Managing Environment Variables for more information.

Note

Any BuildConfig objects created as part of new-app processing will not be updated with environment variables passed via the -e|--env or --env-file argument.

2.2.2.4.2. Specifying Build Environment Variables

When generating applications from a template, source, or an image, you can use the --build-env argument to pass environment variables to the build container at run time:

$ oc new-app openshift/ruby-23-centos7 \
    --build-env HTTP_PROXY=http://myproxy.net:1337/ \
    --build-env GEM_HOME=~/.gem

The variables can also be read from a file using the --build-env-file argument:

$ cat ruby.env
HTTP_PROXY=http://myproxy.net:1337/
GEM_HOME=~/.gem
$ oc new-app openshift/ruby-23-centos7 --build-env-file=ruby.env

Additionally, environment variables can be given on standard input by using --build-env-file=-:

$ cat ruby.env | oc new-app openshift/ruby-23-centos7 --build-env-file=-
2.2.2.4.3. Specifying Labels

When generating applications from source, images, or templates, you can use the -l|--label argument to add labels to the created objects. Labels make it easy to collectively select, configure, and delete objects associated with the application.

$ oc new-app https://github.com/openshift/ruby-hello-world -l name=hello-world
2.2.2.4.4. Viewing the Output Without Creation

To see a dry-run of what new-app will create, you can use the -o|--output argument with a yaml or json value. You can then use the output to preview the objects that will be created, or redirect it to a file that you can edit. Once you are satisfied, you can use oc create to create the OpenShift Container Platform objects.

To output new-app artifacts to a file, edit them, then create them:

$ oc new-app https://github.com/openshift/ruby-hello-world \
    -o yaml > myapp.yaml
$ vi myapp.yaml
$ oc create -f myapp.yaml
2.2.2.4.5. Creating Objects With Different Names

Objects created by new-app are normally named after the source repository, or the image used to generate them. You can set the name of the objects produced by adding a --name flag to the command:

$ oc new-app https://github.com/openshift/ruby-hello-world --name=myapp
2.2.2.4.6. Creating Objects in a Different Project

Normally, new-app creates objects in the current project. However, you can create objects in a different project that you have access to using the -n|--namespace argument:

$ oc new-app https://github.com/openshift/ruby-hello-world -n myproject
2.2.2.4.7. Creating Multiple Objects

The new-app command allows creating multiple applications specifying multiple parameters to new-app. Labels specified in the command line apply to all objects created by the single command. Environment variables apply to all components created from source or images.

To create an application from a source repository and a Docker Hub image:

$ oc new-app https://github.com/openshift/ruby-hello-world mysql
Note

If a source code repository and a builder image are specified as separate arguments, new-app uses the builder image as the builder for the source code repository. If this is not the intent, specify the required builder image for the source using the ~ separator.

2.2.2.4.8. Grouping Images and Source in a Single Pod

The new-app command allows deploying multiple images together in a single pod. In order to specify which images to group together, use the + separator. The --group command line argument can also be used to specify the images that should be grouped together. To group the image built from a source repository with other images, specify its builder image in the group:

$ oc new-app ruby+mysql

To deploy an image built from source and an external image together:

$ oc new-app \
    ruby~https://github.com/openshift/ruby-hello-world \
    mysql \
    --group=ruby+mysql
2.2.2.4.9. Searching for Images, Templates, and Other Inputs

To search for images, templates, and other inputs for the oc new-app command, add the --search and --list flags. For example, to find all of the images or templates that include PHP:

$ oc new-app --search php

2.2.3. Creating an Application Using the Web Console

  1. While in the desired project, click Add to Project:

    Add to Project
  2. Select either a builder image from the list of images in your project, or from the service catalog:

    Browse Catalog
    Note

    Only image stream tags that have the builder tag listed in their annotations appear in this list, as demonstrated here:

    kind: "ImageStream"
    apiVersion: "v1"
    metadata:
      name: "ruby"
      creationTimestamp: null
    spec:
      dockerImageRepository: "registry.redhat.io/openshift3/ruby-20-rhel7"
      tags:
        -
          name: "2.0"
          annotations:
            description: "Build and run Ruby 2.0 applications"
            iconClass: "icon-ruby"
            tags: "builder,ruby" 1
            supports: "ruby:2.0,ruby"
            version: "2.0"
    1
    Including builder here ensures this ImageStreamTag appears in the web console as a builder.
  3. Modify the settings in the new application screen to configure the objects to support your application:

    Configuration Wizard

2.3. Promoting Applications Across Environments

2.3.1. Overview

Application promotion means moving an application through various runtime environments, typically with an increasing level of maturity. For example, an application might start out in a development environment, then be promoted to a stage environment for further testing, before finally being promoted into a production environment. As changes are introduced in the application, again the changes will start in development and be promoted through stage and production.

The "application" today is more than just the source code written in Java, Perl, Python, etc. It is more now than the static web content, the integration scripts, or the associated configuration for the language specific runtimes for the application. It is more than the application specific archives consumed by those language specific runtimes.

In the context of OpenShift Container Platform and its combined foundation of Kubernetes and Docker, additional application artifacts include:

  • Container images with their rich set of metadata and associated tooling.
  • Environment variables that are injected into containers for application use.
  • API objects (also known as resource definitions; see Core Concepts) of OpenShift Container Platform, which:

    • are injected into containers for application use.
    • dictate how OpenShift Container Platform manages containers and pods.

In examining how to promote applications in OpenShift Container Platform, this topic will:

  • Elaborate on these new artifacts introduced to the application definition.
  • Describe how you can demarcate the different environments for your application promotion pipeline.
  • Discuss methodologies and tools for managing these new artifacts.
  • Provide examples that apply the various concepts, constructs, methodologies, and tools to application promotion.

2.3.2. Application Components

2.3.2.1. API Objects

With regard to OpenShift Container Platform and Kubernetes resource definitions (the items newly introduced to the application inventory), there are a couple of key design points for these API objects that are relevant to revisit when considering the topic of application promotion.

First, as highlighted throughout OpenShift Container Platform documentation, every API object can be expressed via either JSON or YAML, making it easy to manage these resource definitions via traditional source control and scripting.

Also, the API objects are designed such that there are portions of the object which specify the desired state of the system, and other portions which reflect the status or current state of the system. This can be thought of as inputs and outputs. The input portions, when expressed in JSON or YAML, in particular are items that fit naturally as source control managed (SCM) artifacts.

Note

Remember, the input or specification portions of the API objects can be totally static or dynamic in the sense that variable substitution via template processing is possible on instantiation.

The result of these points with respect to API objects is that with their expression as JSON or YAML files, you can treat the configuration of the application as code.

Conceivably, almost any of the API objects may be considered an application artifact by your organization. Listed below are the objects most commonly associated with deploying and managing an application:

BuildConfigs
This is a special case resource in the context of application promotion. While a BuildConfig is certainly a part of the application, especially from a developer’s perspective, typically the BuildConfig is not promoted through the pipeline. It produces the Image that is promoted (along with other items) through the pipeline.
Templates
In terms of application promotion, Templates can serve as the starting point for setting up resources in a given staging environment, especially with the parameterization capabilities. Additional post-instantiation modifications are very conceivable though when applications move through a promotion pipeline. See Scenarios and Examples for more on this.
Routes
These are the most typical resources that differ stage to stage in the application promotion pipeline, as tests against different stages of an application access that application via its Route. Also, remember that you have options with regard to manual specification or auto-generation of host names, as well as the HTTP-level security of the Route.
Services
If reasons exist to avoid Routers and Routes at given application promotion stages (perhaps for simplicity’s sake for individual developers at early stages), an application can be accessed via the Cluster IP address and port. If used, some management of the address and port between stages could be warranted.
Endpoints
Certain application-level services (e.g., database instances in many enterprises) may not be managed by OpenShift Container Platform. If so, then creating those Endpoints yourself, along with the necessary modifications to the associated Service (omitting the selector field on the Service) are activities that are either duplicated or shared between stages (based on how you delineate your environment).
Secrets
The sensitive information encapsulated by Secrets are shared between staging environments when the corresponding entity (either a Service managed by OpenShift Container Platform or an external service managed outside of OpenShift Container Platform) the information pertains to is shared. If there are different versions of the said entity in different stages of your application promotion pipeline, it may be necessary to maintain a distinct Secret in each stage of the pipeline or to make modifications to it as it traverses through the pipeline. Also, take care that if you are storing the Secret as JSON or YAML in an SCM, some form of encryption to protect the sensitive information may be warranted.
DeploymentConfigs
This object is the primary resource for defining and scoping the environment for a given application promotion pipeline stage; it controls how your application starts up. While there are aspects of it that will be common across all the different stage, undoubtedly there will be modifications to this object as it progresses through your application promotion pipeline to reflect differences in the environments for each stage, or changes in behavior of the system to facilitate testing of the different scenarios your application must support.
ImageStreams, ImageStreamTags, and ImageStreamImage
Detailed in the Images and Image Streams sections, these objects are central to the OpenShift Container Platform additions around managing container images.
ServiceAccounts and RoleBindings
Management of permissions to other API objects within OpenShift Container Platform, as well as the external services, are intrinsic to managing your application. Similar to Secrets, the ServiceAccounts and RoleBindings objects can vary in how they are shared between the different stages of your application promotion pipeline based on your needs to share or isolate those different environments.
PersistentVolumeClaims
Relevant to stateful services like databases, how much these are shared between your different application promotion stages directly correlates to how your organization shares or isolates the copies of your application data.
ConfigMaps
A useful decoupling of Pod configuration from the Pod itself (think of an environment variable style configuration), these can either be shared by the various staging environments when consistent Pod behavior is desired. They can also be modified between stages to alter Pod behavior (usually as different aspects of the application are vetted at different stages).

2.3.2.2. Images

As noted earlier, container images are now artifacts of your application. In fact, of the new applications artifacts, images and the management of images are the key pieces with respect to application promotion. In some cases, an image might encapsulate the entirety of your application, and the application promotion flow consists solely of managing the image.

Images are not typically managed in a SCM system, just as application binaries were not in previous systems. However, just as with binaries, installable artifacts and corresponding repositories (such as RPMs, RPM repositories, or Nexus) arose with similar semantics to SCMs. Therefore, constructs and terminology around image management that are similar to SCMs have emerged:

  • Image registry == SCM server
  • Image repository == SCM repository

As images reside in registries, application promotion is concerned with ensuring the appropriate image exists in a registry that can be accessed from the environment that needs to run the application represented by that image.

Rather than reference images directly, application definitions typically abstract the reference into an image stream. This means the image stream will be another API object that makes up the application components. For more details on image streams, see Core Concepts.

2.3.2.3. Summary

Now that the application artifacts of note, images and API objects, have been detailed in the context of application promotion within OpenShift Container Platform, the notion of where you run your application in the various stages of your promotion pipeline is next the point of discussion.

2.3.3. Deployment Environments

A deployment environment, in this context, describes a distinct space for an application to run during a particular stage of a CI/CD pipeline. Typical environments include development, test, stage, and production, for example. The boundaries of an environment can be defined in different ways, such as:

  • Via labels and unique naming within a single project.
  • Via distinct projects within a cluster.
  • Via distinct clusters.

And it is conceivable that your organization leverages all three.

2.3.3.1. Considerations

Typically, you will consider the following heuristics in how you structure the deployment environments:

  • How much resource sharing the various stages of your promotion flow allow
  • How much isolation the various stages of your promotion flow require
  • How centrally located (or geographically dispersed) the various stages of your promotion flow are

Also, some important reminders on how OpenShift Container Platform clusters and projects relate to image registries:

  • Multiple project in the same cluster can access the same image streams.
  • Multiple clusters can access the same external registries.
  • Clusters can only share a registry if the OpenShift Container Platform internal image registry is exposed via a route.

2.3.3.2. Summary

After deployment environments are defined, promotion flows with delineation of stages within a pipeline can be implemented. The methods and tools for constructing those promotion flow implementations are the next point of discussion.

2.3.4. Methods and Tools

Fundamentally, application promotion is a process of moving the aforementioned application components from one environment to another. The following subsections outline tools that can be used to move the various components by hand, before advancing to discuss holistic solutions for automating application promotion.

Note

There are a number of insertion points available during both the build and deployment processes. They are defined within BuildConfig and DeploymentConfig API objects. These hooks allow for the invocation of custom scripts which can interact with deployed components such as databases, and with the OpenShift Container Platform cluster itself.

Therefore, it is possible to use these hooks to perform component management operations that effectively move applications between environments, for example by performing an image tag operation from within a hook. However, the various hook points are best suited to managing an application’s lifecycle within a given environment (for example, using them to perform database schema migrations when a new version of the application is deployed), rather than to move application components between environments.

2.3.4.1. Managing API Objects

Resources, as defined in one environment, will be exported as JSON or YAML file content in preparation for importing it into a new environment. Therefore, the expression of API objects as JSON or YAML serves as the unit of work as you promote API objects through your application pipeline. The oc CLI is used to export and import this content.

Tip

While not required for promotion flows with OpenShift Container Platform, with the JSON or YAML stored in files, you can consider storing and retrieving the content from a SCM system. This allows you to leverage the versioning related capabilities of the SCM, including the creation of branches, and the assignment of and query on various labels or tags associated to versions.

2.3.4.1.1. Exporting API Object State

API object specifications should be captured with oc get --export. This operation removes environment specific data from the object definitions (e.g., current namespace or assigned IP addresses), allowing them to be recreated in different environments (unlike oc get operations, which output an unfiltered state of the object).

Use of oc label, which allows for adding, modifying, or removing labels on API objects, can prove useful as you organize the set of object collected for promotion flows, because labels allow for selection and management of groups of pods in a single operation. This makes it easier to export the correct set of objects and, because the labels will carry forward when the objects are created in a new environment, they also make for easier management of the application components in each environment.

Note

API objects often contain references such as a DeploymentConfig that references a Secret. When moving an API object from one environment to another, you must ensure that such references are also moved to the new environment.

Similarly, API objects such as a DeploymentConfig often contain references to ImageStreams that reference an external registry. When moving an API object from one environment to another, you must ensure such references are resolvable within the new environment, meaning that the reference must be resolvable and the ImageStream must reference an accessible registry in the new environment. See Moving Images and Promotion Caveats for more detail.

2.3.4.1.2. Importing API Object State
2.3.4.1.2.1. Initial Creation

The first time an application is being introduced into a new environment, it is sufficient to take the JSON or YAML expressing the specifications of your API objects and run oc create to create them in the appropriate environment. When using oc create, keep the --save-config option in mind. Saving configuration elements on the object in its annotation list facilitates the later use of oc apply to modify the object.

2.3.4.1.2.2. Iterative Modification

After the various staging environments are initially established, as promotion cycles commence and the application moves from stage to stage, the updates to your application can include modification of the API objects that are part of the application. Changes in these API objects are conceivable since they represent the configuration for the OpenShift Container Platform system. Motivations for such changes include:

  • Accounting for environmental differences between staging environments.
  • Verifying various scenarios your application supports.

Transfer of the API objects to the next stage’s environment is accomplished via use of the oc CLI. While a rich set of oc commands which modify API objects exist, this topic focuses on oc apply, which computes and applies differences between objects.

Specifically, you can view oc apply as a three-way merge that takes in files or stdin as the input along with an existing object definition. It performs a three-way merge between:

  1. the input into the command,
  2. the current version of the object, and
  3. the most recent user specified object definition stored as an annotation in the current object.

The existing object is then updated with the result.

If further customization of the API objects is necessary, as in the case when the objects are not expected to be identical between the source and target environments, oc commands such as oc set can be used to modify the object after applying the latest object definitions from the upstream environment.

Some specific usages are cited in Scenarios and Examples.

2.3.4.2. Managing Images and Image Streams

Images in OpenShift Container Platform are managed via a series of API objects as well. However, managing images are so central to application promotion that discussion of the tools and API objects most directly tied to images warrant separate discussion. Both manual and automated forms exist to assist you in managing image promotion (the propagation of images through your pipeline).

2.3.4.2.1. Moving Images
Note

For all the detailed caveats around managing images, refer to the Managing Images topic.

2.3.4.2.1.1. When Staging Environments Share a Registry

When your staging environments share the same OpenShift Container Platform registry, for example if they are all on the same OpenShift Container Platform cluster, there are two operations that are the basic means of moving your images between the stages of your application promotion pipeline:

  1. First, analogous to docker tag and git tag, the oc tag command allows you to update an OpenShift Container Platform image stream with a reference to a specific image. It also allows you to copy references to specific versions of an image from one image stream to another, even across different projects in a cluster.
  2. Second, the oc import-image serves as a bridge between external registries and image streams. It imports the metadata for a given image from the registry and stores it into the image stream as an image stream tag. Various BuildConfigs and DeploymentConfigs in your project can reference those specific images.
2.3.4.2.1.2. When Staging Environments Use Different Registries

More advanced usage occurs when your staging environments leverage different OpenShift Container Platform registries.

Accessing the Internal Registry spells out the steps in detail, but in summary you can:

  1. Use the docker command in conjunction which obtaining the OpenShift Container Platform access token to supply into your docker login command.
  2. After being logged into the OpenShift Container Platform registry, use docker pull, docker tag and docker push to transfer the image.
  3. After the image is available in the registry of the next environment of your pipeline, use oc tag as needed to populate any image streams.
2.3.4.2.2. Deploying

Whether changing the underlying application image or the API objects that configure the application, a deployment is typically necessary to pick up the promoted changes. If the images for your application change (for example, due to an oc tag operation or a docker push as part of promoting an image from an upstream environment), ImageChangeTriggers on your DeploymentConfig can trigger the new deployment. Similarly, if the DeploymentConfig API object itself is being changed, a ConfigChangeTrigger can initiate a deployment when the API object is updated by the promotion step (for example, oc apply).

Otherwise, the oc commands that facilitate manual deployment include:

  • oc rollout: The new approach to manage deployments, including pause and resume semantics and richer features around managing history.
  • oc rollback: Allows for reversion to a previous deployment; in the promotion scenario, if testing of a new version encounters issues, confirming it still works with the previous version could be warranted.
2.3.4.2.3. Automating Promotion Flows with Jenkins

After you understand the components of your application that need to be moved between environments when promoting it and the steps required to move the components, you can start to orchestrate and automate the workflow. OpenShift Container Platform provides a Jenkins image and plug-ins to help with this process.

The OpenShift Container Platform Jenkins image is detailed in Using Images, including the set of OpenShift Container Platform-centric plug-ins that facilitate the integration of Jenkins, and Jenkins Pipelines. Also, the Pipeline build strategy facilitates the integration between Jenkins Pipelines and OpenShift Container Platform. All of these focus on enabling various aspects of CI/CD, including application promotion.

When moving beyond manual execution of application promotion steps, the Jenkins-related features provided by OpenShift Container Platform should be kept in mind:

  • OpenShift Container Platform provides a Jenkins image that is heavily customized to greatly ease deployment in an OpenShift Container Platform cluster.
  • The Jenkins image contains the OpenShift Pipeline plug-in, which provides building blocks for implementing promotion workflows. These building blocks include the triggering of Jenkins jobs as image streams change, as well as the triggering of builds and deployments within those jobs.
  • BuildConfigs employing the OpenShift Container Platform Jenkins Pipeline build strategy enable execution of Jenkinsfile-based Jenkins Pipeline jobs. Pipeline jobs are the strategic direction within Jenkins for complex promotion flows and can leverage the steps provided by the OpenShift Pipeline Plug-in.
2.3.4.2.4. Promotion Caveats
2.3.4.2.4.1. API Object References

API objects can reference other objects. A common use for this is to have a DeploymentConfig that references an image stream, but other reference relationships may also exist.

When copying an API object from one environment to another, it is critical that all references can still be resolved in the target environment. There are a few reference scenarios to consider:

  • The reference is "local" to the project. In this case, the referenced object resides in the same project as the object that references it. Typically the correct thing to do is to ensure that you copy the referenced object into the target environment in the same project as the object referencing it.
  • The reference is to an object in another project. This is typical when an image stream in a shared project is used by multiple application projects. In this case, when copying the referencing object to the new environment, you must update the reference as needed so it can be resolved in the target environment. That may mean:

    • Changing the project the reference points to, if the shared project has a different name in the target environment.
    • Moving the referenced object from the shared project into the local project in the target environment and updating the reference to point to the local project when moving the primary object into the target environment.
    • Some other combination of copying the referenced object into the target environment and updating references to it.

In general, the guidance is to consider objects referenced by the objects being copied to a new environment and ensure the references are resolvable in the target environment. If not, take appropriate action to fix the references and make the referenced objects available in the target environment.

2.3.4.2.4.2. Image Registry References

Image streams point to image repositories to indicate the source of the image they represent. When an image stream is moved from one environment to another, it is important to consider whether the registry and repository reference should also change:

  • If different image registries are used to assert isolation between a test environment and a production environment.
  • If different image repositories are used to separate test and production-ready images.

If either of these are the case, the image stream must be modified when it is copied from the source environment to the target environment so that it resolves to the correct image. This is in addition to performing the steps described in Scenarios and Examples to copy the image from one registry and repository to another.

2.3.4.3. Summary

At this point, the following have been defined:

  • New application artifacts that make up a deployed application.
  • Correlation of application promotion activities to tools and concepts provided by OpenShift Container Platform.
  • Integration between OpenShift Container Platform and the CI/CD pipeline engine Jenkins.

Putting together examples of application promotion flows within OpenShift Container Platform is the final step for this topic.

2.3.5. Scenarios and Examples

Having defined the new application artifact components introduced by the Docker, Kubernetes, and OpenShift Container Platform ecosystems, this section covers how to promote those components between environments using the mechanisms and tools provided by OpenShift Container Platform.

Of the components making up an application, the image is the primary artifact of note. Taking that premise and extending it to application promotion, the core, fundamental application promotion pattern is image promotion, where the unit of work is the image. The vast majority of application promotion scenarios entails management and propagation of the image through the promotion pipeline.

Simpler scenarios solely deal with managing and propagating the image through the pipeline. As the promotion scenarios broaden in scope, the other application artifacts, most notably the API objects, are included in the inventory of items managed and propagated through the pipeline.

This topic lays out some specific examples around promoting images as well as API objects, using both manual and automated approaches. But first, note the following on setting up the environment(s) for your application promotion pipeline.

2.3.5.1. Setting up for Promotion

After you have completed development of the initial revision of your application, the next logical step is to package up the contents of the application so that you can transfer to the subsequent staging environments of your promotion pipeline.

  1. First, group all the API objects you view as transferable and apply a common label to them:

    labels:
      promotion-group: <application_name>

    As previously described, the oc label command facilitates the management of labels with your various API objects.

    Tip

    If you initially define your API objects in a OpenShift Container Platform template, you can easily ensure all related objects have the common label you will use to query on when exporting in preparation for a promotion.

  2. You can leverage that label on subsequent queries. For example, consider the following set of oc command invocations that would then achieve the transfer of your application’s API objects:

    $ oc login <source_environment>
    $ oc project <source_project>
    $ oc get -o yaml --export dc,is,svc,route,secret,sa -l promotion-group=<application_name> > export.yaml
    $ oc login <target_environment>
    $ oc new-project <target_project> 1
    $ oc create -f export.yaml
    1
    Alternatively, oc project <target_project> if it already exists.
    Note

    On the oc get --export command, whether or not you include the is type for image streams depends on how you choose to manage images, image streams, and registries across the different environments in your pipeline. The caveats around this are discussed below. See also the Managing Images topic.

  3. You must also get any tokens necessary to operate against each registry used in the different staging environments in your promotion pipeline. For each environment:

    1. Log in to the environment:

      $ oc login <each_environment_with_a_unique_registry>
    2. Get the access token with:

      $ oc whoami -t
    3. Copy and paste the token value for later use.

2.3.5.2. Repeatable Promotion Process

After the initial setup of the different staging environments for your pipeline, a set of repeatable steps to validate each iteration of your application through the promotion pipeline can commence. These basic steps are taken each time the image or API objects in the source environment are changed:

Move updated images → Move updated API objects → Apply environment specific customizations

  1. Typically, the first step is promoting any updates to the image(s) associated with your application to the next stage in the pipeline. As noted above, the key differentiator in promoting images is whether the OpenShift Container Platform registry is shared or not between staging environments.

    1. If the registry is shared, simply leverage oc tag:

      $ oc tag <project_for_stage_N>/<imagestream_name_for_stage_N>:<tag_for_stage_N> <project_for_stage_N+1>/<imagestream_name_for_stage_N+1>:<tag_for_stage_N+1>
    2. If the registry is not shared, you can leverage the access tokens for each of your promotion pipeline registries as you log in to both the source and destination registries, pulling, tagging, and pushing your application images accordingly:

      1. Log in to the source environment registry:

        $ docker login -u <username> -e <any_email_address> -p <token_value> <src_env_registry_ip>:<port>
      2. Pull your application’s image:

        $ docker pull <src_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
      3. Tag your application’s image to the destination registry’s location, updating namespace, name, and tag as needed to conform to the destination staging environment:

        $ docker tag <src_env_registry_ip>:<port>/<namespace>/<image name>:<tag> <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
      4. Log in to the destination staging environment registry:

        $ docker login -u <username> -e <any_email_address> -p <token_value> <dest_env_registry_ip>:<port>
      5. Push the image to its destination:

        $ docker push <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
        Tip

        To automatically import new versions of an image from an external registry, the oc tag command has a --scheduled option. If used, the image the ImageStreamTag references will be periodically pulled from the registry hosting the image.

  2. Next, there are the cases where the evolution of your application necessitates fundamental changes to your API objects or additions and deletions from the set of API objects that make up the application. When such evolution in your application’s API objects occurs, the OpenShift Container Platform CLI provides a broad range of options to transfer to changes from one staging environment to the next.

    1. Start in the same fashion as you did when you initially set up your promotion pipeline:

      $ oc login <source_environment>
      $ oc project <source_project>
      $ oc get -o yaml --export dc,is,svc,route,secret,sa -l promotion-group=<application_name> > export.yaml
      $ oc login <target_environment>
      $ oc <target_project>
    2. Rather than simply creating the resources in the new environment, update them. You can do this a few different ways:

      1. The more conservative approach is to leverage oc apply and merge the new changes to each API object in the target environment. In doing so, you can --dry-run=true option and examine the resulting objects prior to actually changing the objects:

        $ oc apply -f export.yaml --dry-run=true

        If satisfied, actually run the apply command:

        $ oc apply -f export.yaml

        The apply command optionally takes additional arguments that help with more complicated scenarios. See oc apply --help for more details.

      2. Alternatively, the simpler but more aggressive approach is to leverage oc replace. There is no dry run with this update and replace. In the most basic form, this involves executing:

        $ oc replace -f export.yaml

        As with apply, replace optionally takes additional arguments for more sophisticated behavior. See oc replace --help for more details.

  3. The previous steps automatically handle new API objects that were introduced, but if API objects were deleted from the source environment, they must be manually deleted from the target environment using oc delete.
  4. Tuning of the environment variables cited on any of the API objects may be necessary as the desired values for those may differ between staging environments. For this, use oc set env:

    $ oc set env <api_object_type>/<api_object_ID> <env_var_name>=<env_var_value>
  5. Finally, trigger a new deployment of the updated application using the oc rollout command or one of the other mechanisms discussed in the Deployments section above.

2.3.5.3. Repeatable Promotion Process Using Jenkins

The OpenShift Sample job defined in the Jenkins Docker Image for OpenShift Container Platform is an example of image promotion within OpenShift Container Platform within the constructs of Jenkins. Setup for this sample is located in the OpenShift Origin source repository.

This sample includes:

  • Use of Jenkins as the CI/CD engine.
  • Use of the OpenShift Pipeline plug-in for Jenkins. This plug-in provides a subset of the functionality provided by the oc CLI for OpenShift Container Platform packaged as Jenkins Freestyle and DSL Job steps. Note that the oc binary is also included in the Jenkins Docker Image for OpenShift Container Platform, and can also be used to interact with OpenShift Container Platform in Jenkins jobs.
  • The OpenShift Container Platform-provided templates for Jenkins. There is a template for both ephemeral and persistent storage.
  • A sample application: defined in the OpenShift Origin source repository, this application leverages ImageStreams, ImageChangeTriggers, ImageStreamTags, BuildConfigs, and separate DeploymentConfigs and Services corresponding to different stages in the promotion pipeline.

The following examines the various pieces of the OpenShift Sample job in more detail:

  1. The first step is the equivalent of an oc scale dc frontend --replicas=0 call. This step is intended to bring down any previous versions of the application image that may be running.
  2. The second step is the equivalent of an oc start-build frontend call.
  3. The third step is the equivalent of an oc rollout latest dc/frontend call.
  4. The fourth step is the "test" for this sample. It ensures that the associated service for this application is in fact accessible from a network perspective. Under the covers, a socket connection is attempted against the IP address and port associated with the OpenShift Container Platform service. Of course, additional tests can be added (if not via OpenShift Pipepline plug-in steps, then via use of the Jenkins Shell step to leverage OS-level commands and scripts to test your application).
  5. The fifth step commences under that assumption that the testing of your application passed and hence intends to mark the image as "ready". In this step, a new prod tag is created for the application image off of the latest image. With the frontend DeploymentConfig having an ImageChangeTriggerdefined for that tag, the corresponding "production" deployment is launched.
  6. The sixth and last step is a verification step, where the plug-in confirms that OpenShift Container Platform launched the desired number of replicas for the "production" deployment.