How to open a PROACTIVE case for patching or upgrading Red Hat OpenShift Container Platform

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)

Issue

  • Upgrading production OpenShift Container Platform clusters.
  • How to open a case to notify Red Hat of upgrades taking place so appropriate resources are available?
  • What information is useful for Red Hat support when opening a proactive ticket for weekend or after hours work?

NOTE: For new installations it is not necessary to create a proactive case. For those situations, simply raise a regular support ticket if any problem arises, please also note that new installations can't be considered severity 1 (urgent) accordingly with our Red Hat Support Severity Level Definitions.

Resolution

Introduction - what is a Proactive Case?

Proactive cases are a way for Premium customers to provide an advance notice to Red Hat Support of planned upgrades, migrations, patching, or any other maintenance activity. Typically, these events happen outside regular business hours.
Having prior knowledge of the planned activity provides Red Hat with an opportunity to ensure the right information is made available to better prepare technical support should you encounter an issue.

Note: New installations do not entitle a Proactive Case, as they do not fit with the Severity Level Definitions for Severity 1. For issues encountered during new installations, please open a regular Support Case.

Prerequisites

  • The account must have an active Premium OpenShift subscription.
    • If your account has an active Standard: Red Hat Cloud Suite, Standard (2 Sockets, 32 Cores) subscription, or you have integrated ARO (Azure Red Hat OpenShift) as a Partner Product (for which the Microsoft provides the subscription and not Red Hat) then you are entitled to submit and engage in a Proactive Case
  • It is recommended to submit Proactive Cases several days before the activity is scheduled to begin (preferably 7-14 days in advance), in order to allocate appropriate resources for the upcoming event.
    • If the activity requires weekend coverage, the Proactive Case must be submitted at least 4 business days before the upcoming activity (by close of the business day, for example on the Tuesday prior to the upcoming activity).
    • Note: Weekend Proactive Cases submitted less than 4 business days before the activity will be considered "Late Entry", and treated through normal Follow the Sun workflow. For such, Red Hat will not be able to guarantee that the appropriate resources would be staffed for the weekend.
  • In order to allocate the most appropriate engineer from different teams, no matter where they may be worldwide, Proactive Support Cases are conducted in the English language only.
  • It is recommended to familiarize yourself with the Reference Guide for Engaging with Red Hat Support article, which provides best practices for engaging Red Hat Support.
  • Prepare all relevant details for the upcoming event, that will help communicate what changes and activities are taking place.
  • If you have a Technical Account Manager (TAM), a Solution Architect (SA), or a Customer Success Manager (CSM) - please engage them when planning a Proactive Case.
  • Check the overall state of the cluster following OpenShift 4 cluster upgrade pre-checks requirements.
  • Review Navigating Kubernetes API deprecations and removals for getting information about the Kubernetes deprecated APIs used in the cluster.

Opening a Proactive Support Case

Open a Support Case and provide the required information in the following manner:

  • Case Title - should be prefixed with "[Proactive]" and should summarize the version upgrade. Example:

    [PROACTIVE] Production/Integration/Development OpenShift cluster upgrade from 3.X to 3.Y
    
  • Severity - open the case on a Severity level of 3. On the day of activity or when the activity begins, Red Hat recommend the customers to reset the Severity to 1 so that the proper resources would be allocated to assist in case any issue arises.

  • Case Description - describe as many details as possible, for example:
    • Specific activity occurring during the event.
    • Environment.
    • Infrastructure.
    • System(s) involved.
    • Other Components being upgraded.
  • Date / Time of Event - state the planned start and end date/time of the maintenance window duration in UTC format.
  • Contact Information - provide the primary and secondary contact name, email addresses and phone number with country and region code. This is especially important if the person who opened the case is not the person doing the maintenance activity.
  • Data Attachments - Collect must-gather logs and if needed sosreports (refer to the Diagnostic Steps section below) of all the systems involved before any upgrade work is performed.
    • When collecting sosreports if there are only a few systems, please attach them to the case. Otherwise, please keep them locally and put a note in the case description confirming that you have captured all of them. This will help to prevent possible issues that were already there and identify new issues after the upgrade.
    • Collect the information about the Kubernetes deprecated APIs used in the cluster following Navigating Kubernetes API deprecations and removals.

NOTE: Unless exceptionally recommended by your TAM or CSM (if you have), it is not advisable to update more than one minor version at a time, if in spite of this you still need to upgrade more than one version in the same maintenance window, please keep in mind that for workload reasons we can't guarantee a dedicated resource in place.

Example of a Proactive Case:

  • Case Title:

    [PROACTIVE] Production OpenShift cluster upgrade from 3.9 to 3.10
    
  • Case Description:

    In two weeks, on Saturday July 22nd,  we are planning to upgrade our production cluster from OCP X.Y.Z to OCP X2.Y2.Z2 on RHEL7.5.
    These guest systems are installed on Red Hat Virtualization 4.2. All systems have 8 vCPU's and 32GB of RAM. No memory or CPU over commitment is being used.
    
    Cluster details:
      3 MASTERS.
      3 ETCD NODES.
      3 INFRA NODES.
      20 APP NODES.
      50 PROJECTS.
      200 SERVICES.
      400 PODS.
    
    Maintenance Window Information:
       - Date - Sunday, July 22nd
       - Time frame - 6 hours
       - Timezone - CEST
       - Start Time - 08:00 AM UTC+2
       - End Time - 14:00 PM UTC+2
    
    Primary Contact Information:
       - Name: Jane Doe
       - Email: jane.doe@example.com
       - Phone: +1-123-456-7890
    
    Secondary Contact Information:
      - Name: John Doe
      - Email: john.doe@example.com
      - Phone: +1-123-456-7891
    
  • Case Attachments:

    must-gather.tar.gz
    sos-report.tar.gz
    

Diagnostic Steps

OCP 3

  • To properly collect a sosreports on OCP 3.x, you can follow the instructions below or refer to the link:

    $ sosreport -e docker -k docker.all=on -k docker.logs=on -k origin.diag=off
    
  • When using Azure Red Hat OpenShift (ARO), please capture an sosreport using the instructions described in this article.

OCP 4

  • Please extract a must-gather log bundle containing a full picture of the cluster general status and configurations. You can refer to the instructions below or to the Documentation link

    • Navigate to the directory where you want to store the must-gather data.
    • Run the oc adm must-gather command:
    $ oc adm must-gather
    
    • Create a compressed file from the must-gather directory that was just created in your working directory. For example, on a computer that uses a Linux operating system, run the following command:
    $ tar cvaf must-gather.tar.gz must-gather.local.5421342344627712289/ 
    
    • Make sure to replace must-gather-local.5421342344627712289/ with the actual directory name & Attach the compressed file to your support case on the Red Hat Customer Portal.
    • If requested, sosreports can also be extract from the RHCOS underlying nodes within OCP4. If needed, please check this solution.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments