Chapter 2. Downloading log files and diagnostic information using must-gather

If Red Hat OpenShift Container Storage is unable to automatically resolve a problem, use the must-gather tool to collect log files and diagnostic information so that you or Red Hat support can review the problem and determine a solution.

Important

When OpenShift Container Storage is deployed in external mode, must-gather only collects logs from the Red Hat OpenShift Container Storage cluster and does not collect debug data and logs from the external Red Hat Ceph Storage cluster. To collect debug logs from the external Red Hat Ceph Storage cluster, see Red Hat Ceph Storage Troubleshooting guide and contact your Red Hat Ceph Storage Administrator.

Procedure

  • Run the must-gather command from the client connected to the OpenShift Container Storage cluster:

    $ oc adm must-gather --image=registry.redhat.io/ocs4/ocs-must-gather-rhel8:v4.6 --dest-dir=<directory-name>

    This collects the following information in the specified directory:

    • All OpenShift Container Storage cluster related Custom Resources (CRs) with their namespaces.
    • Pod logs of all the OpenShift Container Storage related pods.
    • Output of some standard Ceph commands like Status, Cluster health, and others.

Command variations

  • If one or more master nodes are not in the Ready state, use --node-name to provide a master node that is Ready so that the must-gather pod can be safely scheduled.

    $ oc adm must-gather --image=registry.redhat.io/ocs4/ocs-must-gather-rhel8:v4.6 --dest-dir=<directory-name> --node-name=<node-name>
  • If you want to gather information from a specific time:

    • To specify a relative time period for logs gathered, such as within 5 seconds or 2 days, add /usr/bin/gather since=<duration>:

      $ oc adm must-gather --image=registry.redhat.io/ocs4/ocs-must-gather-rhel8:v4.6 --dest-dir=<directory-name> /usr/bin/gather since=<duration>
    • To specify a specific time to gather logs after, add /usr/bin/gather since-time=<rfc3339-timestamp>:

      $ oc adm must-gather --image=registry.redhat.io/ocs4/ocs-must-gather-rhel8:v4.6 --dest-dir=<directory-name> /usr/bin/gather since-time=<rfc3339-timestamp>

    Replace the example values in these commands as follows:

    <node-name>
    If one or more master nodes are not in the Ready state, use this parameter to provide the name of a master node that is still in the Ready state. This avoids scheduling errors by ensuring that the must-gather pod is not scheduled on a master node that is not ready.
    <directory-name>
    The directory to store information collected by must-gather.
    <duration>
    Specify the period of time to collect information from as a relative duration, for example, 5h (starting from 5 hours ago).
    <rfc3339-timestamp>
    Specify the period of time to collect information from as an RFC 3339 timestamp, for example, 2020-11-10T04:00:00+00:00 (starting from 4am UTC on 11 Nov 2020).