OpenShift infrastructure components can be installed across multiple hosts. The following sections outline the system requirements and instructions for preparing your environment and hosts before installing OpenShift.

You must have an active OpenShift Enterprise subscription on your Red Hat account to proceed. If you do not, contact your sales representative for more information.

The system requirements vary per host type:

Persistent Storage

The Kubernetes persistent volume framework allows you to provision an OpenShift cluster with persistent storage using networked storage available in your environment. This can be done after completing the initial OpenShift installation depending on your application needs, giving users a way to request those resources without having any knowledge of the underlying infrastructure.

Currently NFS is fully supported, however other options are available as Technology Preview.

Configuring Core Usage

By default, OpenShift masters and nodes use all available cores in the system they run on. You can choose the number of cores you want OpenShift to use by setting the GOMAXPROCS environment variable.

For example, run the following before starting the server to make OpenShift only run on one core:

# export GOMAXPROCS=1

Security Warning

OpenShift runs Docker containers on your hosts, and in some cases, such as build operations and the registry service, it does so using privileged containers. Furthermore, those containers access your host’s Docker daemon and perform docker build and docker push operations. As such, you should be aware of the inherent security risks associated with performing docker run operations on arbitrary images as they effectively have root access.

For more information, see these articles:

To address these risks, OpenShift uses security context constraints that control the actions that pods can perform and what it has the ability to access.

DNS

A wildcard for a DNS zone must ultimately resolve to the IP address of the OpenShift router.

For example, create a wildcard DNS entry for cloudapps, or something similar, that has a low TTL and points to the public IP address of the host where the router will be deployed:

*.cloudapps.example.com. 300 IN  A 192.168.133.2

In almost all cases, when referencing VMs you must use host names, and the host names that you use must match the output of the hostname -f command on each node.

Networking

A shared network must exist between the master and node hosts.

If you plan to configure multiple masters for high-availability using the advanced installation method, you must also select an IP to be configured as your virtual IP (VIP) during the installation process. The IP that you select must be routable between all of your nodes, and if you configure using a FQDN it should resolve on all nodes. The VIP is then managed by Pacemaker.

Git

You must have either Internet access and a GitHub account, or read and write access to an internal, HTTP-based Git server.

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Before installing OpenShift, you must first prepare each host per the following.

The quick and advanced installation methods require a user that has access to all hosts. If you want to run the installer as a non-root user, passwordless sudo rights must be configured on each destination host.

For example, you can generate an SSH key on the host where you will invoke the installation process:

# ssh-keygen

Do not use a password.

An easy way to distribute your SSH keys is by using a bash loop:

# for host in master.example.com \
    node1.example.com \
    node2.example.com; \
    do ssh-copy-id -i ~/.ssh/id_rsa.pub $host; \
    done

Modify the host names in the above command according to your configuration.

Now that your environment and hosts are properly set up, you can install OpenShift Enterprise using the quick installation or advanced installation method.

The quick installation method allows you to use an interactive CLI utility to install OpenShift across a set of hosts. The installation utility can deploy OpenShift components on targeted hosts by installing RPMs.

This installation method is provided to make the installation experience easier by interactively gathering the data needed to run on each host. The utility is a self-contained wrapper intended for usage on a Red Hat Enterprise Linux (RHEL) 7 system.

In addition to running interactive installations from scratch, the atomic-openshift-installer command can also be run or re-run using a predefined installation configuration file. This file can be used with the installation utility to:

Alternatively, you can use the advanced installation method for more complex environments.

The installation utility allows you to install OpenShift master and node components on a defined set of hosts.

Before installing OpenShift, you must first satisfy the prerequisites on your hosts, which includes verifying system and environment requirements and properly installing and configuring Docker. You must also be prepared to provide or validate the following information for each of your targeted hosts during the course of the installation:

After following the instructions in the Prerequisites topic, you can continue to running an interactive or unattended installation.

You can start the interactive installation by running:

$ atomic-openshift-installer install

Then follow the on-screen instructions to install a new OpenShift Enterprise cluster.

After it has finished, ensure that you back up the ~/.config/openshift/installer.cfg.ymlinstallation configuration file that is created, as it is required if you later want to re-run the installation, add hosts to the cluster, or upgrade your cluster. Then, see What’s Next for the next steps on configuring your OpenShift cluster.

The installation utility can use a predefined installation configuration file, which contains information about your installation, individual hosts, and cluster. When running an interactive installation, an installation configuration file based on your answers is created for you in ~/.config/openshift/installer.cfg.yml. The file is created if you are instructed to exit the installation to manually modify the configuration or when the installation completes. You can also create the configuration file manually from scratch to perform an unattended installation.

version: v1 1
variant: openshift-enterprise 2
variant_version: 3.0 3
ansible_ssh_user: root 4
ansible_log_path: /tmp/ansible.log 5
hosts: 6
- ip: 10.0.0.1 7
  hostname: master-private.example.com 8
  public_ip: 24.222.0.1 9
  public_hostname: master.example.com 10
  master: true 11
  node: true 12
  connect_to: 24.222.0.1 13
- ip: 10.0.0.2
  hostname: node1-private.example.com
  public_ip: 24.222.0.2
  public_hostname: node1.example.com
  node: true
  connect_to: 10.0.0.2
- ip: 10.0.0.3
  hostname: node2-private.example.com
  public_ip: 24.222.0.3
  public_hostname: node2.example.com
  node: true
  connect_to: 10.0.0.3

Unattended installations allow you to define your hosts and cluster configuration in an installation configuration file before running the installation utility so that you do not have to go through all of the interactive installation questions and answers. It also allows you to resume an interactive installation you may have left unfinished, and quickly get back to where you left off.

To run an unattended installation, first define an installation configuration file at ~/.config/openshift/installer.cfg.yml. Then, run the installation utility with the -u flag:

$ atomic-openshift-installer -u install

By default in interactive or unattended mode, the installation utility uses the configuration file located at ~/.config/openshift/installer.cfg.yml if the file exists. If it does not exist, attempting to start an unattended installation fails. Alternatively, you can specify a different location for the configuration file using the -c option, but doing so will require you to specify the file location every time you run the installation:

$ atomic-openshift-installer -u -c </path/to/file> install

After the unattended installation finishes, ensure that you back up the ~/.config/openshift/installer.cfg.yml file that was used, as it is required if you later want to re-run the installation, add hosts to the cluster, or upgrade your cluster. Then, see What’s Next for the next steps on configuring your OpenShift cluster.

Whether you began the process using an interactive or unattended installation, you can re-run the installation as long as you have an installation configuration file at ~/.config/openshift/installer.cfg.yml (or specify its location with the -c option).

To re-run an installation, use the install subcommand again in interactive or unattended mode:

$ atomic-openshift-installer install

The installer will detect your installed environment and allow you to either add an additional node or perform a clean install:

Gathering information from hosts...
Installed environment detected.
By default the installer only adds new nodes to an installed environment.
Do you want to (1) only add additional nodes or (2) perform a clean install?:

Choose one of the options and follow the on-screen instructions to complete your desired task.

You can uninstall OpenShift using the installation utility by running:

$ atomic-openshift-installer uninstall

Now that you have a working OpenShift Enterprise instance, you can:

For production environments, a reference configuration implemented using Ansible playbooks is available as the advanced installation method for installing OpenShift hosts. Familiarity with Ansible is assumed, however you can use this configuration as a reference to create your own implementation using the configuration management tool of your choosing.

Alternatively, you can use the quick installation method for trial installations.

Before installing OpenShift, you must first see the Prerequisites topic to prepare your hosts, which includes verifying system and environment requirements per component type and properly installing and configuring Docker. It also includes installing Ansible version 1.8.4 or later, as the advanced installation method is based on Ansible playbooks and as such requires directly invoking Ansible.

After following the instructions in the Prerequisites topic, you can continue to Configuring Ansible.

The /etc/ansible/hosts file is Ansible’s inventory file for the playbook to use during the installation. The inventory file describes the configuration for your OpenShift cluster. You must replace the default contents of the file with your desired configuration.

The following sections describe commonly-used variables to set in your inventory file during an advanced installation, followed by example inventory files you can use as a starting point for your installation. The examples describe various environment topographies. You can choose an example that matches your requirements, modify it to match your own environment, and use it as your inventory file when running the Ansible installer.

Configuring Host Variables

To assign environment variables to hosts during the Ansible install, indicate the desired variables in the /etc/ansible/hosts file after the host entry in the [masters] or [nodes] sections. For example:

[masters]
ec2-52-6-179-239.compute-1.amazonaws.com openshift_public_hostname=ose3-master.public.example.com

The following table describes variables for use with the Ansible installer that can be assigned to individual host entries:

Configuring Cluster Variables

To assign environment variables during the Ansible install that apply more globally to your OpenShift cluster overall, indicate the desired variables in the /etc/ansible/hosts file on separate, single lines within the [OSEv3:vars] section. For example:

[OSEv3:vars]

openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/openshift/openshift-passwd'}]

osm_default_subdomain=apps.test.example.com

The following table describes variables for use with the Ansible installer that can be assigned cluster-wide:

Configuring Node Host Labels

You can assign labels to node hosts during the Ansible install by configuring the /etc/ansible/hosts file. Labels are useful for determining the placement of pods onto nodes using the scheduler.

To assign labels to a node host during an Ansible install, use the openshift_node_labels variable with the desired labels added to the desired node host entry in the [nodes] section. For example:

[nodes]
node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"

# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes

# Set variables common for all OSEv3 hosts
[OSEv3:vars]
# SSH user, this user should allow ssh based auth without requiring a password
ansible_ssh_user=root

# If ansible_ssh_user is not root, ansible_sudo must be set to true
#ansible_sudo=true

product_type=openshift
deployment_type=enterprise

# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider
#openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/openshift/openshift-passwd'}]

# host group for masters
[masters]
master.example.com

# host group for nodes, includes region info
[nodes]
master.example.com openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"
node2.example.com openshift_node_labels="{'region': 'primary', 'zone': 'west'}"

# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
etcd

# Set variables common for all OSEv3 hosts
[OSEv3:vars]
ansible_ssh_user=root
product_type=openshift
deployment_type=enterprise

# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider
#openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/openshift/openshift-passwd'}]

# host group for masters
[masters]
master.example.com

# host group for etcd
[etcd]
etcd1.example.com
etcd2.example.com
etcd3.example.com

# host group for nodes, includes region info
[nodes]
master.example.com openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"
node2.example.com openshift_node_labels="{'region': 'primary', 'zone': 'west'}"

# Create an OSEv3 group that contains the masters and nodes groups
[OSEv3:children]
masters
nodes
etcd

# Set variables common for all OSEv3 hosts
[OSEv3:vars]
ansible_ssh_user=root
product_type=openshift
deployment_type=enterprise

# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider
# openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/openshift/openshift-passwd'}]

# master cluster ha variables using pacemaker or RHEL HA
openshift_master_cluster_method=pacemaker
openshift_master_cluster_password=openshift_cluster
openshift_master_cluster_vip=192.168.133.25
openshift_master_cluster_public_vip=192.168.133.25
openshift_master_cluster_hostname=openshift-master.example.com
openshift_master_cluster_public_hostname=openshift-master.example.com


# host group for masters
[masters]
master1.example.com
master2.example.com
master3.example.com

# host group for etcd
[etcd]
etcd1.example.com
etcd2.example.com
etcd3.example.com

# host group for nodes, includes region info
[nodes]
master[1:3].example.com openshift_node_labels="{'region': 'infra', 'zone': 'default'}"
node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"
node2.example.com openshift_node_labels="{'region': 'primary', 'zone': 'west'}"

After you’ve configured Ansible by defining an inventory file in /etc/ansible/hosts, you can run the Ansible installer:

# ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml

If for any reason the installation fails, before re-running the installer, see Known Issues to check for any specific instructions or workarounds.

If you installed OpenShift using a configuration for multiple masters, you must configure a fencing device. See Fencing: Configuring STONITH in the High Availability Add-on for Red Hat Enterprise Linux documentation for instructions, then continue to Verifying the Installation.

After the installer completes, you can verify that the master is started and nodes are registered and reporting in Ready status by running the following as root:

# oc get nodes

NAME                      LABELS                                                                     STATUS
master.example.com        kubernetes.io/hostname=master.example.com,region=infra,zone=default        Ready,SchedulingDisabled
node1.example.com         kubernetes.io/hostname=node1.example.com,region=primary,zone=east          Ready
node2.example.com         kubernetes.io/hostname=node2.example.com,region=primary,zone=west          Ready

Multiple etcd Hosts

If you installed multiple etcd hosts:

Multiple Masters

If you installed multiple masters:

Red Hat recommends that you also verify your installation by consulting the High Availability Add-on for Red Hat Enterprise Linux documentation.

After your cluster is installed, you can install additional nodes (including masters) and add them to your cluster by running the scaleup.yml playbook. This playbook queries the master, generates and distributes new certificates for the new nodes, then runs the configuration playbooks on the new nodes only.

This process is similar to re-running the installer in the quick installation method to add nodes, however you have more configuration options available when using the advanced method and running the playbooks directly.

You must have an existing inventory file (for example, /etc/ansible/hosts) that is representative of your current cluster configuration in order to run the scaleup.yml playbook. If you previously used the atomic-openshift-installer command to run your installation, you can check ~/.config/openshift/.ansible/hosts for the last inventory file that the installer generated and use or modify that as needed as your inventory file. You must then specify the file location with -i when calling ansible-playbook later.

To add nodes to an existing cluster:

The following are known issues for specified installation configurations.

Multiple Masters

Now that you have a working OpenShift instance, you can:

OpenShift can build Docker images from your source code, deploy them, and manage their lifecycle. To enable this, OpenShift provides an internal, integrated Docker registry that can be deployed in your OpenShift environment to locally manage images.

To deploy the integrated Docker registry, use the oadm registry command as a user with cluster administrator privileges. For example:

$ oadm registry --config=/etc/openshift/master/admin.kubeconfig \1
    --credentials=/etc/openshift/master/openshift-registry.kubeconfig \2
    --images='registry.access.redhat.com/openshift3/ose-${component}:${version}' 3

This creates a service and a deployment configuration, both called docker-registry. Once deployed successfully, a pod is created with a name similar to docker-registry-1-cpty9.

To see a full list of options that you can specify when creating the registry:

$ oadm registry --help

$ oc volume deploymentconfigs/docker-registry --add --name=v1 -t pvc \
     --claim-name=<pvc_name> --overwrite

$ oc volume deploymentconfigs/docker-registry \
     --add --overwrite --name=registry-storage --mount-path=/registry \
     --source='{"nfs": { "server": "<fqdn>", "path": "/path/to/export"}}'

To view the logs for the Docker registry, run the oc logs indicating the desired pod:

$ oc logs docker-registry-1-da73t
2015-05-01T19:48:36.300593110Z time="2015-05-01T19:48:36Z" level=info msg="version=v2.0.0+unknown"
2015-05-01T19:48:36.303294724Z time="2015-05-01T19:48:36Z" level=info msg="redis not configured" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002
2015-05-01T19:48:36.303422845Z time="2015-05-01T19:48:36Z" level=info msg="using inmemory layerinfo cache" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002
2015-05-01T19:48:36.303433991Z time="2015-05-01T19:48:36Z" level=info msg="Using OpenShift Auth handler"
2015-05-01T19:48:36.303439084Z time="2015-05-01T19:48:36Z" level=info msg="listening on :5000" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002

Tag and image metadata is stored in OpenShift, but the registry stores layer and signature data in a volume that is mounted into the registry container at /registry. As oc exec does not work on privileged containers, to view a registry’s contents you must manually SSH into the node housing the registry pod’s container, then run docker exec on the container itself:

To access the registry directly, such as to perform docker push or docker pull operations, you must first log in to the registry using an access token.

You can now perform docker pull and docker push operations against your registry. For example:

Optionally, you can secure the registry so that it serves traffic via TLS:

To expose your internal registry externally, it is recommended that you run a secure registry. To expose the registry you must first have deployed a router.

After you have a registry deployed, you can:

The OpenShift router is the ingress point for all external traffic destined for services in your OpenShift installation. OpenShift provides and supports the following two router plug-ins:

Starting in OpenShift Enterprise 3.0.1.0, you must first create a service account for the router before deploying. This service account must have permissions to a security context constraint (SCC) that allows it to specify host ports.

Create a service account, for example named router:

$ echo \
    '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"router"}}' \
    | oc create -f -

Edit the privileged SCC:

$ oc edit scc privileged

Add the router service account in the form of system:serviceaccount:<project>:<name> to the users section:

...
users:
- system:serviceaccount:openshift-infra:build-controller
- system:serviceaccount:default:router

The oadm router command is provided with the administrator CLI to simplify the tasks of setting up routers in a new installation. Just about every form of communication between OpenShift components is secured by TLS and uses various certificates and authentication methods. Use the --credentials option to specify what credentials the router should use to contact the master.

First, ensure you have created the router service account before deploying a router.

To check if a default router, named router, already exists:

$ oadm router --dry-run \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

To see what the default router would look like if created:

$ oadm router -o yaml \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

To create a router if it does not exist:

$ oadm router <router_name> --replicas=<number> \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

Multiple instances are created on different hosts according to the scheduler policy.

To use a different router image and view the router configuration that would be used:

$ oadm router <router_name> -o <format> --images=<image> \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

For example:

$ oadm router region-west -o yaml --images=myrepo/somerouter:mytag \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

routingConfig:
  subdomain: v3.openshift.test

myroute-mynamespace.v3.openshift.test

$ CA=/etc/openshift/master
$ oadm ca create-server-cert --signer-cert=$CA/ca.crt \
      --signer-key=$CA/ca.key --signer-serial=$CA/ca.serial.txt \
      --hostnames='*.cloudapps.example.com' \
      --cert=cloudapps.crt --key=cloudapps.key

$ cat cloudapps.crt cloudapps.key $CA/ca.crt > cloudapps.router.pem

$ oadm router --default-cert=cloudapps.router.pem \
    --credentials="$KUBECONFIG" --service-account=router

# openssl rsa -in <passwordProtectedKey.key> -out <new.key>

# oadm router --replicas=1 --credentials=$KUBECONFIG --service-account=router

# sudo openssl genrsa -out example-test.key 2048
#
# sudo openssl req -new -key example-test.key -out example-test.csr  \
  -subj "/C=US/ST=CA/L=Mountain View/O=OS3/OU=Eng/CN=www.example.test"
#
# sudo openssl x509 -req -days 366 -in example-test.csr  \
      -signkey example-test.key -out example-test.crt

# servicename="my-service"
# echo "
apiVersion: v1
kind: Route
metadata:
  name:  secured-edge-route
spec:
  host: www.example.test
  to:
    kind: Service
    name: $servicename
  tls:
    termination: edge
    key: |
$(openssl rsa -in example-test.key | sed 's/^/      /')
    certificate: |
$(openssl x509 -in example-test.crt | sed 's/^/      /')

" > example-test-route.yaml

# oc create -f example-test-route.yaml

# routerip="4.1.1.1"  #  replace with IP address of one of your router instances.
# curl -k --resolve www.example.test:443:$routerip https://www.example.test/

$ oadm router \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router \
    --host-network=false

The HAProxy router is based on a golang template that generates the HAProxy configuration file from a list of routes. If you want a customized template router to meet your needs, you can customize the template file, build a new Docker image, and run a customized router.

One common case for this might be implementing new features within the application back ends. For example, it might be desirable in a highly-available setup to use stick-tables that synchronizes between peers. The router plug-in provides all the facilities necessary to make this customization.

You can obtain a new haproxy-config.template file from the latest router image by running:

# docker run --rm --interactive=true --tty --entrypoint=cat \
    registry.access.redhat.com/openshift3/ose-haproxy-router:v3.0.2.0 haproxy-config.template

Save this content to a file for use as the basis of your customized template.

{{ if (len .PeerEndpoints) gt 0 }}
peers openshift_peers
  {{ range $endpointID, $endpoint := .PeerEndpoints }}
    peer {{$endpoint.TargetName}} {{$endpoint.IP}}:1937
  {{ end }}
{{ end }}

peer_name=$HOSTNAME 1

if [ -n "$old_pid" ]; then
  /usr/sbin/haproxy -f $config_file -p $pid_file -L $peer_name -sf $old_pid
else
  /usr/sbin/haproxy -f $config_file -p $pid_file -L $peer_name
fi

            {{ if eq $cfg.TLSTermination "passthrough" }}
backend be_tcp_{{$cfgIdx}}
  balance leastconn
  timeout check 5000ms
  stick-table type ip size 1m expire 5m{{ if (len $.PeerEndpoints) gt 0 }} peers openshift_peers {{ end }}
  stick on src
                {{ range $endpointID, $endpoint := $serviceUnit.EndpointTable }}
  server {{$endpointID}} {{$endpoint.IP}}:{{$endpoint.Port}} check inter 5000ms
                {{ end }}
            {{ end }}

The F5 router plug-in is provided as a Docker image and run as a pod, just like the default HAProxy router. Deploying the F5 router is done similarly as well, using the oadm router command but providing additional flags (or environment variables) to specify the following parameters for the F5 BIG-IP® host:

As with the HAProxy router, the oadm router command creates the service and deployment configuration objects, and thus the replication controllers and pod(s) in which the F5 router itself runs. The replication controller restarts the F5 router in case of crashes. Because the F5 router is only watching routes and endpoints and configuring F5 BIG-IP® accordingly, running the F5 router in this way along with an appropriately configured F5 BIG-IP® deployment should satisfy high-availability requirements.

To deploy the F5 router:

$ oadm router --external-host-partition-path=/OpenShift/zone1 ...

If you deployed an HAProxy router, you can learn more about monitoring the router.

If you have not yet done so, you can:

You can populate your OpenShift installation with a useful set of Red Hat-provided image streams and templates to make it easy for developers to create new applications. By default, the quick installation and advanced installation methods automatically create these sets in the openshift project, which is a default project to which all users have view access.

Use the following instructions to create the objects yourself. The files are installed on the file system of your master.

The core set of image streams provide images that can be used to build Node.js, Perl, PHP, Python, and Ruby applications. It also defines images for MongoDB, MySQL, and PostgreSQL to support data storage.

If your node hosts are subscribed using Red Hat Subscription Manager and you want to use the Red Hat Enterprise Linux (RHEL) 7 based images:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/image-streams/image-streams-rhel7.json \
    -n openshift

Alternatively, to create the core set of image streams that use the CentOS 7 based images:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/image-streams/image-streams-centos7.json \
    -n openshift

It is not possible to create both the CentOS and RHEL sets of image streams because they use the same names. If you desire to have both sets of image streams available to users, either create one set in a different project, or edit one of the files and modify the image stream names to make them unique.

The xPaaS Middleware image streams provide images for JBoss EAP, JBoss JWS, and JBoss A-MQ. They can be used to build applications for those platforms using the provided templates.

To create the xPaaS Middleware set of image streams:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/xpaas-streams/jboss-image-streams.json \
    -n openshift

The database service templates make it easy to run a database image which can be utilized by other components. For each database (MongoDB, MySQL, and PostgreSQL), two templates are defined.

One template uses ephemeral storage in the container which means data stored will be lost if the container is restarted, for example if the pod moves. This template should be used for demonstration purposes only.

The other template defines a persistent volume for storage, however it requires your OpenShift installation to have persistent volumes configured.

To create the core set of database templates:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/db-templates -n openshift

After creating the templates, users are able to easily instantiate the various templates, giving them quick access to a database deployment.

The InstantApp templates define a full set of objects for a running application. These include:

Some of the templates also define a database deployment and service so the application can perform database operations.

After creating the templates, users are able to easily instantiate full applications using the various language images provided with OpenShift. They can also customize the template parameters during instantiation so that it builds source from their own repository rather than the sample repository, so this provides a simple starting point for building new applications.

To create the core InstantApp templates:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/quickstart-templates -n openshift

There is also a set of templates for creating applications using various xPaaS Middleware products (JBoss EAP, JBoss JWS, and JBoss A-MQ), which can be registered by running:

$ oc create -f \
    openshift-ansible/roles/openshift_examples/files/examples/xpaas-templates -n openshift

With these artifacts created, developers can now log into the web console and follow the flow for creating from a template. Any of the database or application templates can be selected to create a running database service or application in the current project. Note that some of the application templates define their own database services as well.

The example applications are all built out of GitHub repositories which are referenced in the templates by default, as seen in the SOURCE_REPOSITORY_URL parameter value. Those repositories can be forked, and the fork can be provided as the SOURCE_REPOSITORY_URL parameter value when creating from the templates. This allows developers to experiment with creating their own applications.

You can direct your developers to the Using the InstantApp Templates section in the Developer Guide for these instructions.

Upgrade your masters first. On each master host, upgrade the openshift-master package:

# yum upgrade openshift-master

Then, restart the openshift-master service and review its logs to ensure services have been restarted successfully:

# systemctl restart openshift-master
# journalctl -r -u openshift-master

After a cluster upgrade, the recommended default cluster roles may have been updated. To check if an update is recommended for your environment, you can run:

# oadm policy reconcile-cluster-roles

This command outputs a list of roles that are out of date and their new proposed values. For example:

# oadm policy reconcile-cluster-roles
apiVersion: v1
items:
- apiVersion: v1
  kind: ClusterRole
  metadata:
    creationTimestamp: null
    name: admin
  rules:
  - attributeRestrictions: null
    resources:
    - builds/custom
...

You can either modify this output to re-apply any local policy changes you have made, or you can automatically apply the new policy by running:

# oadm policy reconcile-cluster-roles --confirm

After upgrading your masters, you can upgrade your nodes. When restarting the openshift-node service, there will be a brief disruption of outbound network connectivity from running pods to services while the service proxy is restarted. The length of this disruption should be very short and scales based on the number of services in the entire cluster.

On each node host, upgrade all openshift packages:

# yum upgrade openshift\*

Then, restart the openshift-node service:

# systemctl restart openshift-node

As a user with cluster-admin privileges, verify that all nodes are showing as Ready:

# oc get nodes
NAME                    LABELS                                        STATUS
master.example.com      kubernetes.io/hostname=master.example.com     Ready,SchedulingDisabled
node1.example.com       kubernetes.io/hostname=node1.example.com      Ready
node2.example.com       kubernetes.io/hostname=node2.example.com      Ready

If you have previously deployed a router, the router deployment configuration must be upgraded to apply updates contained in the router image. To upgrade your router without disrupting services, you must have previously deployed a highly-available routing service.

Edit your router’s deployment configuration. For example, if it has the default router name:

# oc edit dc/router

Apply the following changes:

...
spec:
 template:
    spec:
      containers:
      - env:
        ...
        image: registry.access.redhat.com/openshift3/ose-haproxy-router:v3.0.2.0 1
        imagePullPolicy: IfNotPresent
        ...

You should see one router pod updated and then the next.

The registry must also be upgraded for changes to take effect in the registry image. If you have used a PersistentVolumeClaim or a host mount point, you may restart the registry without losing the contents of your registry. The registry installation topic details how to configure persistent storage.

Edit your registry’s deployment configuration:

# oc edit dc/docker-registry

Apply the following changes:

...
spec:
 template:
    spec:
      containers:
      - env:
        ...
        image: registry.access.redhat.com/openshift3/ose-docker-registry:v3.0.2.0 1
        imagePullPolicy: IfNotPresent
        ...

By default, the quick installation and advanced installation methods automatically create default image streams, QuickStart templates, and database service templates in the openshift project, which is a default project to which all users have view access. These objects were created during installation from the JSON files located under /usr/share/openshift/examples. Running the latest installer will copy newer files into place, but it does not currently update the openshift project.

You can update the openshift project by running the following commands. It is expected that you will receive warnings about items that already exist.

# oc create -n openshift -f /usr/share/openshift/examples/image-streams/image-streams-rhel7.json
# oc create -n openshift -f /usr/share/openshift/examples/db-templates
# oc create -n openshift -f /usr/share/openshift/examples/quickstart-templates
# oc create -n openshift -f /usr/share/openshift/examples/xpaas-streams
# oc create -n openshift -f /usr/share/openshift/examples/xpaas-templates
# oc replace -n openshift -f /usr/share/openshift/examples/image-streams/image-streams-rhel7.json
# oc replace -n openshift -f /usr/share/openshift/examples/db-templates
# oc replace -n openshift -f /usr/share/openshift/examples/quickstart-templates
# oc replace -n openshift -f /usr/share/openshift/examples/xpaas-streams
# oc replace -n openshift -f /usr/share/openshift/examples/xpaas-templates

After updating the default image streams, you may also want to ensure that the images within those streams are updated. For each image stream in the default openshift project, you can run:

# oc import-image -n openshift <imagestream>

For example, get the list of all image streams in the default openshift project:

# oc get is -n openshift
NAME     DOCKER REPO                                                      TAGS                   UPDATED
mongodb  registry.access.redhat.com/openshift3/mongodb-24-rhel7           2.4,latest,v3.0.0.0    16 hours ago
mysql    registry.access.redhat.com/openshift3/mysql-55-rhel7             5.5,latest,v3.0.0.0    16 hours ago
nodejs   registry.access.redhat.com/openshift3/nodejs-010-rhel7           0.10,latest,v3.0.0.0   16 hours ago
...

Update each image stream one at a time:

# oc import-image -n openshift nodejs
Waiting for the import to complete, CTRL+C to stop waiting.
The import completed successfully.

Name:                   nodejs
Created:                16 hours ago
Labels:                 <none>
Annotations:            openshift.io/image.dockerRepositoryCheck=2015-07-21T13:17:00Z
Docker Pull Spec:       registry.access.redhat.com/openshift3/nodejs-010-rhel7

Tag             Spec            Created         PullSpec                                                        Image
0.10            latest          16 hours ago    registry.access.redhat.com/openshift3/nodejs-010-rhel7:latest   66d92cebc0e48e4e4be3a93d0f9bd54f21af7928ceaa384d20800f6e6fcf669f
latest                          16 hours ago    registry.access.redhat.com/openshift3/nodejs-010-rhel7:latest   66d92cebc0e48e4e4be3a93d0f9bd54f21af7928ceaa384d20800f6e6fcf669f
v3.0.0.0        <pushed>        16 hours ago    registry.access.redhat.com/openshift3/nodejs-010-rhel7:v3.0.0.0 66d92cebc0e48e4e4be3a93d0f9bd54f21af7928ceaa384d20800f6e6fcf669f

The following steps are required for the OpenShift Enterprise 3.0.1.0 release.

Creating a Service Account for the Router

The default HAProxy router was updated to utilize host ports and requires that a service account be created and made a member of the privileged security context constraint (SCC). Additionally, "down-then-up" rolling upgrades have been added and is now the preferred strategy for upgrading routers.

After upgrading your master and nodes but before updating to the newer router, you must create a service account for the router. As a cluster administrator, ensure you are operating on the default project:

# oc project default

Delete any existing router service account and create a new one:

# oc delete serviceaccount/router
serviceaccounts/router

# echo '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"router"}}' | oc create -f -
serviceaccounts/router

Edit the privileged SCC:

# oc edit scc privileged

Apply the following changes:

allowHostDirVolumePlugin: true
allowHostNetwork: true 1
allowHostPorts: true 2
allowPrivilegedContainer: true
...
users:
- system:serviceaccount:openshift-infra:build-controller
- system:serviceaccount:default:router 3

Edit your router’s deployment configuration:

# oc edit dc/router

Apply the following changes:

...
spec:
  replicas: 2
  selector:
    router: router
  strategy:
    resources: {}
    rollingParams:
      intervalSeconds: 1
      timeoutSeconds: 120
      updatePeriodSeconds: 1
      updatePercent: -10 1
    type: Rolling
    ...
  template:
    ...
    spec:
      ...
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      serviceAccount: router 2
      serviceAccountName: router 3
...

Now upgrade your router per the standard router upgrade steps.

The following steps are required for the OpenShift Enterprise 3.0.2.0 release.

Switching the Router to Use the Host Network Stack

The default HAProxy router was updated to use the host networking stack by default instead of the former behavior of using the container network stack, which proxied traffic to the router, which in turn proxied the traffic to the target service and container. This new default behavior benefits performance because network traffic from remote clients no longer needs to take multiple hops through user space in order to reach the target service and container.

Additionally, the new default behavior enables the router to get the actual source IP address of the remote connection. This is useful for defining ingress rules based on the originating IP, supporting sticky sessions, and monitoring traffic, among other uses.

Existing router deployments will continue to use the container network stack unless modified to switch to using the host network stack.

To switch the router to use the host network stack, edit your router’s deployment configuration:

# oc edit dc/router

Apply the following changes:

...
spec:
  replicas: 2
  selector:
    router: router
    ...
  template:
    ...
    spec:
      ...
      ports:
        - containerPort: 80 1
          hostPort: 80
          protocol: TCP
        - containerPort: 443 2
          hostPort: 443
          protocol: TCP
        - containerPort: 1936 3
          hostPort: 1936
          name: stats
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
      dnsPolicy: ClusterFirst
      hostNetwork: true 4
      restartPolicy: Always
...

Now upgrade your router per the standard router upgrade steps.

Configuring serviceNetworkCIDR for the SDN

Add the serviceNetworkCIDR parameter to the networkConfig section in /etc/openshift/master/master-config.yaml. This value should match the servicesSubnet value in the kubernetesMasterConfig section:

kubernetesMasterConfig:
  servicesSubnet: 172.30.0.0/16
...
networkConfig:
  serviceNetworkCIDR: 172.30.0.0/16

Adding the Scheduler Configuration API Version

The scheduler configuration file incorrectly lacked kind and apiVersion fields when deployed using the quick or advanced installation methods. This will affect future upgrades, so it is important to add those values if they do not exist.

Modify the /etc/openshift/master/scheduler.json file to add the kind and apiVersion fields:

{
  "kind": "Policy", 1
  "apiVersion": "v1", 2
  "predicates": [
  ...
}