Planning your deployment

Red Hat OpenShift Container Storage 4.3

Important considerations when deploying RHOCS 4.3

Red Hat Storage Documentation Team

Abstract

Read this document for important considerations when planning your Red Hat OpenShift Container Storage deployment.

Chapter 1. Introduction to Red Hat OpenShift Container Storage

Red Hat OpenShift Container Storage is software-defined storage that is optimised for container environments. It runs as an operator on Red Hat OpenShift Container Platform to provide highly integrated and simplified persistent storage management for containers.

Red Hat OpenShift Container Storage supports a variety of storage types, including:

  • Block storage for databases
  • Shared file storage for continuous integration, messaging, and data aggregation
  • Object storage for archival, backup, and media storage

Red Hat OpenShift Container Storage version 4.2 onwards uses:

  • Red Hat Ceph Storage to provide the file, block, and object storage that backs persistent volumes
  • Rook.io to manage and orchestrate provisioning of persistent volumes and claims.
  • NooBaa provides object storage, and its Multicloud Object Gateway enables data federation across multiple cloud environments. NooBaa’s enterprise version is named Red Hat Multicloud Object Gateway.

Chapter 2. Architecture of OpenShift Container Storage

Red Hat OpenShift Container Storage uses Red Hat OpenShift Container Platform as a base. For information about the architecture and lifecycle of OpenShift Container Platform, see OpenShift Container Platform architecture.

Red Hat OpenShift Container Storage architecture

Red Hat OpenShift Container Storage architecture

Red Hat OpenShift Container Storage comprises of 3 main operators, which codify administrative tasks and custom resources so that task and resource characteristics can be more easily automated. Administrators define the desired end state of the cluster, and various operators work to ensure the cluster is either in that state, or approaching that state, with minimal administrator intervention.

The OpenShift Container Storage (OCS) operator
A meta-operator that codifies and enforces the recommendations and requirements of a supported Red Hat OpenShift Container Storage deployment by drawing on other operators in specific, tested ways. This operator provides the storage cluster resource that wraps resources provided by the Rook-Ceph and NooBaa operators.
The Rook-Ceph operator
This operator automates the packaging, deployment, management, upgrading, and scaling of persistent storage provided to container applications, and infrastructure services provided to OpenShift Container Platform. It provides the Ceph cluster resource, which manages the pods that host services such as the Object Storage Daemons, monitors, and the metadata server for the Ceph file system.
The NooBaa operator
This operator provides the Multicloud Object Gateway, an S3 compatible object store service that allows resource access across multiple cloud environments.

2.1. Supported workload types

Red Hat OpenShift Container Storage provides storage appropriate for a number of workload types.

Block storage is suitable for databases and other low-latency transactional workloads. Some examples of supported workloads are Red Hat OpenShift Container Platform logging and monitoring, and PostgreSQL.

Object storage is for video and audio files, compressed data archives, and the data used to train artificial intelligence or machine learning programs. In addition, object storage can be used for any application developed with a cloud-first approach.

File storage is for continuous integration and delivery, web application file storage, and artificial intelligence or machine learning data aggregation. Supported workloads include Red Hat OpenShift Container Platform registry and messaging using JBoss AMQ.

Chapter 3. Supported infrastructure and platforms

Red Hat OpenShift Container Storage supports deployment on Red Hat OpenShift Container Platform deployed with Installer Provisioned Infrastructure (Full Stack Automation) and User Provisioned Infrastructure (Pre Existing Infrastructure).

  • Installer Provisioned Infrastructure (IPI)

    With full stack automation, the installer controls all areas of the installation including infrastructure provisioning with an opinionated best practices deployment on Red Hat OpenShift Container Platform.

  • User Provisioned Infrastructure (UPI))

    With pre-existing infrastructure deployments, administrators are responsible for creating and managing their own infrastructure allowing greater customization and operational flexibility.

Red Hat OpenShift Container Storage supports deployment on the following infrastructures:

Table 3.1. Minimum infrastructure requirements

InfrastructureDeployment typeMinimum requirements

Amazon Web Services

IPI, UPI

  • Amazon EC2 with

    • an instance type m5.4xlarge
    • i3en instance type i3en.2xlarge (Technology preview)

See Creating OpenShift Container Storage cluster on Amazon EC2 for more details.

VMware

UPI

vSphere 6.7 Update 2 and higher with vSAN or VMFS datastore.

See VMware vSphere infrastructure requirements for details.

Bare Metal

UPI

Storage type must be SSD (NVMe/SATA/SAS)

(Technology preview)

3.1. Node requirements

The requirements mentioned in the section apply to pre-existing infrastructure deployments only and are required for exclusive use by Red Hat OpenShift Container Storage.

A minimum of 3 nodes are required for deployment with at least one Object Storage Daemon (OSD) and one Monitor daemon (MON) on each node.

Table 3.2. Minimum requirements for each starting node

ComponentsRequirements

CPU

16

Note: 8 CPUs for Amazon EC2 i3en which is currently a technology preview feature.

Memory

64  GB

Disk

  • For dynamic storage deployments,

    • 1 disk of size 0.5 TiB or 2 TiB or 4 TiB storage
  • For Local Storage deployment,

    • any disk size of 4 TiB or lesser can be used.
    • disk partitioning is not supported.
    • maintain uniform disk sizes across nodes for storage disks.

MON

10 GiB storage per MON on all the storage nodes.

  • MON will use 10 GiB storage space under /var/lib/rook for local storage deployments.
  • In case a node with MON fails and the MON failover to a new node, 10 GiB space will then be consumed on the new node as well.

Note: At a time only 3 nodes require the storage space.

Note
  • In case you plan to run any other workload on a storage node, you must add additional resources (CPU/Memory/Space).
  • In this section, 1 CPU Unit maps to the Kubernetes concept of 1 CPU unit. For more information, see CPU units.

    • 1 unit of CPU is equivalent to 1 core for non-hyperthreaded CPUs.
    • 2 units of CPU are equivalent to 1 core for hyperthreaded CPUs.
    • OpenShift Container Storage core-based subscriptions always come in pairs (2 cores).

      Example: For a 3 node cluster, a minimum of 3 x 16 = 48 units of CPU are required. 48 units of CPU are equivalent to 24 cores which is equivalent to 12 quantity of Red Hat OpenShift Container Storage (2 core) subscriptions.

3.2. Platform requirements

Red Hat OpenShift Container Storage is compatible only with the latest Red Hat OpenShift Container Platform versions.

For Red Hat OpenShift Container Storage 4.3, it is recommended to use Red Hat OpenShift Container Platform 4.3.2 and higher for flexibility in deployment.

For more information, see Red Hat OpenShift Container Storage and Red Hat OpenShift Container Platform interoperability matrix.

Important

Red Hat OpenShift Container Platform must not be installed or running with Federal Information Processing Standards (FIPS) mode enabled.

Nodes that run only storage workloads require a subscription for Red Hat OpenShift Container Storage.

Nodes that run other workloads in addition to storage workloads require both Red Hat OpenShift Container Storage and Red Hat OpenShift Container Platform subscriptions.

Chapter 4. Supported configurations

Red Hat OpenShift Container Storage is deployed as a minimal cluster of 3 worker nodes. Spread the nodes across three different availability zones to ensure availability.

4.1. Using dynamically created storage

Red Hat OpenShift Container Storage supports different storage disk sizes with a capacity of 0.5 TiB, 2 TiB and 4 TiB when storage is dynamically created.

4.1.1. Storage class requirements

Red Hat OpenShift Container Storage makes use of the Red Hat OpenShift Container Platform default storage class, and expects a certain default storage class depending on your infrastructure provider.

These classes are configured on Red Hat OpenShift Container Platform nodes automatically, but if your Red Hat OpenShift Container Platform node uses a different storage class as the default, you must change the default storage class back to the appropriate storage class for your infrastructure provider.

  • On Amazon Web Services, the default storage class must be gp2.
  • On VMware vSphere, the default storage class must be thin.

4.1.2. Sizing and scaling for dynamic storage

The initial cluster of 3 nodes can later be expanded to a maximum of 9 nodes that can support up to 27 disks (3 disks on each node). In case of more than 3 worker nodes, the distribution of the disks depends on OpenShift scheduling and available resources.

Expand the cluster in sets of three nodes to ensure that your storage is replicated, and to ensure you can use at least three availability zones.

Note

You can expand the storage capacity only in the increment of the capacity selected at the time of installation.

The following tables shows the supported configurations for Red Hat OpenShift Container Storage.

Table 4.1. Initial configuration across 3 nodes

DisksDisks per nodeTotal capacityUsable storage capacity

0.5 TiB

1

1.5 TiB

0.5 TiB

2 TiB

1

6 TiB

2 TiB

4 TiB

1

12 TiB

4 TiB

Table 4.2. Expanded configuration of up to 9 nodes

Disk size (N)Maximum disks per nodeMaximum total capacity (= 27 disks x N)Maximum usable storage capacity

0.5 TiB

3

13.5 TiB

4.5 TiB

2 TiB

3

54 TiB

18 TiB

4 TiB

3

108 TiB

36 TiB

Warning

Always ensure that you have plenty of storage capacity.

If storage ever fills completely, it is not possible to add capacity or to delete or migrate content away from the storage to free up space. Completely full storage is very difficult to recover.

Capacity alerts are issued when cluster storage capacity reaches 75% (near-full) and 85% (full) of total capacity. Always address capacity warnings promptly, and review your storage regularly to ensure that you do not run out of storage space.

If you do run out of storage space completely, contact Red Hat Customer Support.

As of Red Hat OpenShift Container Storage 4.3, installation is supported only on existing Red Hat OpenShift Container Platform nodes. See Deploying OpenShift Container Storage for more information.

4.2. Using local storage devices

Use of local devices requires the installation of the local storage operator using Operator Hub. See Installing OpenShift Container Storage using local storage devices.

4.2.1. Storage sizing and scaling for local devices

For local storage deployment, any disk size of 4 TiB or lesser can be used, and all disks must be in the same size and type.

The initial cluster of 3 nodes can later be expanded to a maximum of 9 nodes that can support up to 27 disks (3 disks on each node). In case of more than 3 worker nodes, the distribution of the disks depend on OpenShift scheduling and available resources. Maximum you can increase the cluster size to 108 TiB with a usable capacity of 36 TiB upon expansion.

Warning

Always ensure that you have plenty of storage capacity.

If storage ever fills completely, it is not possible to add capacity or to delete or migrate content away from the storage to free up space. Completely full storage is very difficult to recover.

Capacity alerts are issued when cluster storage capacity reaches 75% (near-full) and 85% (full) of total capacity. Always address capacity warnings promptly, and review your storage regularly to ensure that you do not run out of storage space.

If you do run out of storage space completely, contact Red Hat Customer Support.

Chapter 5. Next steps

Go to Deploying OpenShift Container Storage to start deploying your container storage solution.