Procedure 4.2. To Bootstrap the Upgrade and Perform the begin Step:

1. The openshift-enterprise-release RPM package includes the ose-upgrade tool that guides you through the upgrade process. Install the openshift-enterprise-release package on each host, and update it to the most current version.

   # yum install openshift-enterprise-release

2. The begin step of the upgrade process applies to all hosts, and includes those hosts that contain only supporting services such as MongoDB and ActiveMQ. Hosts using Red Hat Subscription Management (RHSM) or Red Hat Network (RHN) Classic are unsubscribed from the 1.2 channels and subscribed to the new 2.0 channels.

   Warning

   This step assumes that the channel names come directly from Red Hat Network. If the package source is an instance of Red Hat Satellite or Subscription Asset Manager and the channel names are remapped differently, you must change this yourself. Examine the scripts in the /usr/lib/ruby/site_ruby/1.8/ose-upgrade/host/upgrades/2/ directory for use as models. You can also add your custom script to a subdirectory to be executed with the ose-upgrade tool.
   In addition to updating the channel set, modifications to the yum configuration give priority to the OpenShift Enterprise, Red Hat Enterprise Linux, and JBoss repositories. However, packages from other sources are excluded as required to prevent certain issues with dependency management that occur between the various channels.
   Run the begin step on each host. Note that the command output is different depending on the type of host. The following example output is from a broker host:

   # ose-upgrade begin
   
   INFO: OpenShift broker installed.
   INFO: Setting host step 'begin' status to UPGRADING
   INFO: Starting upgrade number 2 to version 2.0.
   [...]
   INFO: Setting host step 'begin' status to COMPLETE
   INFO: To continue the upgrade, install a specific upgrade package.

Procedure 4.3. To Install the Upgrade RPM Specific to a Host:

1. Depending on the host type, install the latest upgrade RPM package from the new OpenShift Enterprise 2.0 channels. For broker hosts, install the openshift-enterprise-upgrade-broker package:

   # yum install openshift-enterprise-upgrade-broker

   For node hosts, install the openshift-enterprise-upgrade-node package:

   # yum install openshift-enterprise-upgrade-node

   If the package is already installed because of a previous upgrade, it still must be updated to the latest package version for the OpenShift Enterprise 2.0 upgrade.
2. The ose-upgrade tool guides the upgrade process by listing the necessary steps that are specific to the upgrade scenario, and identifies the step to be performed next. The ose-upgrade status command, or ose-upgrade, provides a current status report. The command output varies depending on the type of host. The following example output is from a broker host:

   # ose-upgrade status
   
   INFO: OpenShift broker installed.
   Current upgrade is number 2 to version 2.0.
   Step sequence:
     begin pre outage rpms conf maintenance_mode pending_ops confirm_nodes data gears end_maintenance_mode post
   Next step is: pre

Procedure 4.4. To Perform the pre Step on Broker and Node Hosts:

1. The pre step manages the following actions:
   • Backs up OpenShift Enterprise configuration files.
   • Clears pending operations older than one hour. (Broker hosts only)
   • Performs any pre-upgrade datastore migration steps. (Broker hosts only)
   • Updates authorization indexes. (Broker hosts only)
   Run the pre step on one broker host and each node host:

   # ose-upgrade pre

   When one broker host begins this step, any attempts made by other broker hosts to run the pre step simultaneously will fail.
2. After the pre step completes on the first broker host, run it on any remaining broker hosts.

Procedure 4.5. To Perform the outage Step on Broker and Node Hosts:

1. The outage step stops services as required depending on the type of host.

   Warning

   The broker enters outage mode during this upgrade step. A substantial outage also begins for applications on the node hosts. Scaled applications are unable to contact any child gears during the outage. These outages last until the end_maintenance_mode step is complete.
   Perform this step on all broker hosts first, and then on all node hosts. This begins the broker outage, and all communication between the broker host and the node hosts is stopped. Perform the outage step with the following command:

   # ose-upgrade outage

   After the command completes on all hosts, node and broker hosts can be upgraded simultaneously until the upgrade steps are complete on all node hosts, and the broker host reaches the confirm_nodes step.
2. For all other hosts that are not a broker or a node host, run yum update to upgrade any services that are installed, such as MongoDB or ActiveMQ:

   # yum update

Procedure 4.6. To Perform the rpms Step on Broker and Node Hosts:

• The rpms step updates RPM packages installed on the node host, and installs any new RPM packages that are required.
  Run the rpms step on each host:

  # ose-upgrade rpms

Procedure 4.7. To Perform the conf Step on Broker and Node Hosts:

• The conf step changes the OpenShift Enterprise configuration to match the new codebase installed in the previous step. Each modified file is first copied to a file with the same name plus a .ugsave extension and a timestamp. This makes it easier to determine what files have changed.
  Run the conf step on each host:

  # ose-upgrade conf

  Warning

  If the configuration files have been significantly modified from the recommended configuration, manual intervention may be required to merge configuration changes so that they can be used with OpenShift Enterprise.

Procedure 4.8. To Perform the maintenance_mode Step on Broker and Node Hosts:

• The maintenance_mode step manages the following actions:
  • Configures the broker to disable the API and return an outage notification to any requests. (Broker hosts only)
  • Starts the broker service and, if installed, the console service in maintenance mode so that they provide clients with an outage notification. (Broker hosts only)
  • Clears the broker and console caches. (Broker hosts only)
  • Enables gear upgrade extensions. (Node hosts only)
  • Starts the ruby193-mcollective service. (Node hosts only)
  Run the maintenance_mode step on each host:

  # ose-upgrade maintenance_mode

Procedure 4.9. To Perform the pending_ops Step on a Broker Host:

1. The pending_ops step clears records of any pending application operations; the outage prevents them from ever completing. Run the pending_ops step on one broker host. Do not run this command on multiple broker hosts at the same time. When one broker host begins this step, any attempts made by other broker hosts to run the pending_ops step simultaneously will fail:

   # ose-upgrade pending_ops

2. After the pending_ops step completes on the first broker host, run the command on any remaining broker hosts.

Procedure 4.10. To Perform the confirm_nodes Step on Broker Hosts:

1. The confirm_nodes step attempts to access all known node hosts to determine whether they have all been upgraded before proceeding. This step fails if the maintenance_mode step has not been completed on all node hosts, or if MCollective cannot access any node hosts.
   Run the confirm_nodes step on a broker host:

   # ose-upgrade confirm_nodes

2. If this step fails due to node hosts that are no longer deployed, you may need to skip the confirm_nodes step. Ensure that all node hosts reported missing are not actually expected to respond, then skip the confirm_nodes step with the following command:

   # ose-upgrade --skip confirm_nodes

Procedure 4.11. To Perform the data Step on Broker Hosts:

1. The data step runs a data migration against the shared broker datastore. Run the data step on one broker host:

   # ose-upgrade data

   When one broker host begins this step, any attempts made by other broker hosts to run the data step simultaneously will fail.
2. After the data step completes on the first broker host, run it on any remaining broker hosts.

Procedure 4.12. To Perform the gears Step on Broker Hosts:

1. The gears step runs a gear migration through the required changes so that they can be used in OpenShift Enterprise 2.0. Run the gears step on one broker host:

   # ose-upgrade gears

   When one broker host begins this step, any attempts made by other broker hosts to run the gears step simultaneously will fail.
2. After the gears step completes on the first broker host, run it on any remaining broker hosts.

Procedure 4.13. To Perform the test_gears_complete Step on Node Hosts:

• The test_gears_complete step verifies the gear migrations are complete before proceeding. This step blocks the upgrade on node hosts by waiting until the gears step has completed on an associated broker host. Run the test_gears_complete step on all node hosts:

  # ose-upgrade test_gears_complete

Procedure 4.14. To Perform the end_maintenance_mode Step on Broker and Node Hosts:

1. The end_maintenance_mode step starts the services that were stopped in the maintenance_mode step or added in the interim. It gracefully restarts httpd to complete the node host upgrade, and restarts the broker service and, if installed, the console service. Complete this step on all node hosts first before running it on the broker hosts:

   # ose-upgrade end_maintenance_mode

2. Run the oo-accept-node script on each node host to verify that it is correctly configured:

   # oo-accept-node

Procedure 4.15. To Perform the post Step on Broker Hosts:

1. The post step manages the following actions on the broker host:
   • Performs any post-upgrade datastore migration steps.
   • Publishes updated district UIDs to the node hosts.
   • Clears the broker and console caches.
   Run the post step on a broker host:

   # ose-upgrade post

   When one broker host begins this step, any attempts made by other broker hosts to run the post step simultaneously will fail.
2. After the post step completes on the first broker host, run it on any remaining broker hosts.
3. The upgrade is now complete for an OpenShift Enterprise installation. Run oo-diagnostics on each host to diagnose any problems:

   # oo-diagnostics