Chapter 1. Components and Considerations

1.1. Red Hat OpenStack Platform Instances

The reference environment in this document consists of the following OpenStack instances:

a single bastion instance
three master instance
six node instance (3 infrastructure nodes and 3 application nodes)

The bastion host serves as the installer of the Ansible playbooks that deploy Red Hat OpenShift Container Platform as well as an entry point for management tasks.

The master nodes contain the master components, including the API server, controller manager server, and etcd. The master manages nodes in its Kubernetes cluster and schedules pods to run on nodes.

The nodes provide the runtime environments for the containers. Each node in a cluster has the required services to be managed by the master nodes. Nodes have the required services to run pods.

For more information about the differences between master and nodes see OpenShift Documentation - Kubernetes Infrastructure.

1.2. Master Instances and Components

Table 1.1. Master Components¹

Component	Description
API Server	The Kubernetes API server validates and configures the data for pods, services, and replication controllers. It also assigns pods to nodes and synchronizes pod information with service configuration.
`etcd`	`etcd` stores the persistent master state while other components watch `etcd` for changes to bring themselves into the desired state. `etcd` can be optionally configured for high availability, typically deployed with 2n+1 peer services.
Controller Manager Server	The controller manager server watches `etcd` for changes to replication controller objects and then uses the API to enforce the desired state. Can be run as a standalone process. Several such processes create a cluster with one active leader at a time.

When using the native high availability method with HAProxy, master components have the following availability.

Table 1.2. Availability Matrix with HAProxy¹

Role	Style	Notes
API Server	Active-Active	Managed by HAProxy
Controller Manager Server	Active-Passive	One instance is elected as a cluster lead at a time

1: OpenShift Documentation - Kubernetes Infrastructure

1.3. Node Instances and Components

The node instances run containers on behalf of its users and are separated into two functional classes: infrastructure and application nodes. The infrastructure (infra) nodes run the OpenShift router , OpenShift logging , OpenShift metrics , and the OpenShift registry while the application (app) nodes host the user container processes.

The Red Hat OpenShift Container Platform SDN requires Master instances to be considered nodes, therefore, all nodes run the atomic-openshift-node service.

The routers are an important component of Red Hat OpenShift Container Platform as they are the entry point to applications running in Red Hat OpenShift Container Platform. Developers can expose their applications running in Red Hat OpenShift Container Platform to be available from outside the Red Hat OpenShift Container Platform SDN creating routes that are published in the routers. Once the traffic reaches the router, the router forwards traffic to the containers on the app nodes using the Red Hat OpenShift Container Platform SDN.

In this reference architecture a set of three routers are deployed on the infra nodes for high availability purposes. In order to have a single entry point for applications, a load balancer is created as part of the installation in the same Red Hat OpenStack Platform tenant where it can reach the infrastructure nodes for load balancing the incoming traffic to the routers running in the infra nodes.

Note

If using an external load balancer, the infra nodes require a public IP address in order to communicate with the load balancer host.

Table 1.3. Node Instance types

Type	Node role
Master	Host web console and service catalog items.
Infrastructure node	Host Red Hat OpenShift Container Platform infrastructure pods (registry, router, logging, metrics,…)
Application node	Host applications deployed on Red Hat OpenShift Container Platform

1.4. Instance Storage

Cinder volumes attach to instances to provide additional disk space. Red Hat OpenShift Container Platform requires at least 40 GB of space available on the master nodes and at least 15GB on the infrastructure and application nodes for the /var partition. This reference architecture breaks out /var/lib/etcd, docker storage, and ephemeral pod storage to provide individual storage for those specific directories.

Each master node Red Hat OpenStack Platform instance creates and mounts cinder volumes using the steps provided in this reference architecture. One volume contains a volume for etcd storage. The master nodes while not schedulable by default, contain cinder volumes for both docker storage and Red Hat OpenShift Container Platform pod storage.

Each infra and app node Red Hat OpenStack Platform instance creates and mounts cinder volumes using the steps provided in this reference architecture. One volume contains the docker runtime disk overhead to store the docker container images. The other cinder volume is created for pod storage. Each node offers cinder access to the containers it hosts through Red Hat OpenShift Container Platform.

Table 1.4. Instances storage

Instance type	Role
Masters only	`etcd` data
Masters, infra and app nodes	Store docker images
Masters, infra and app nodes	Pod local storage

1.5. Load Balancers

This guide uses an external load balancer running haproxy to offer a single entry point for the many Red Hat OpenShift Container Platform components. Organizations can provide their own currently deployed load balancers in the event that the service already exists.

The Red Hat OpenShift Container Platform console, provided by the Red Hat OpenShift Container Platform master nodes, can be spread across multiple instances to provide both load balancing and high availability properties.

Application traffic passes through the Red Hat OpenShift Container Platform Router on its way to the container processes. The Red Hat OpenShift Container Platform Router is a reverse proxy service container that multiplexes the traffic to multiple containers making up a scaled application running inside Red Hat OpenShift Container Platform. The load balancer used by infra nodes acts as the public view for the Red Hat OpenShift Container Platform applications.

The destination for the master and application traffic must be set in the load balancer configuration after each instance is created, the floating IP address is assigned and before the installation. A single haproxy load balancer can forward both sets of traffic to different destinations.

1.6. DNS

DNS service is an important component in the Red Hat OpenShift Container Platform environment. Regardless of the provider of DNS, an organization is required to have certain records in place to serve the various Red Hat OpenShift Container Platform components.

Note

Red Hat OpenStack Platform 10 includes a DNS-as-a-Service (DNSaaS) component, also known as Designate, but it is currently in Technology Preview phase

Since the load balancer values for the Red Hat OpenShift Container Platform master service and infrastructure nodes running router pods are known beforehand, entries should be configured into the DNS prior to starting the deployment procedure.

1.6.1. Application DNS

Applications served by OpenShift are accessible by the router on ports 80/TCP and 443/TCP. The router uses a wildcard record to map all host names under a specific sub domain to the same IP address without requiring a separate record for each name.

This allows Red Hat OpenShift Container Platform to add applications with arbitrary names as long as they are under that sub domain.

For example, a wildcard record for *.apps.example.com causes DNS name lookups for tax.apps.example.com and home-goods.apps.example.com to both return the same IP address: 10.19.x.y. All traffic is forwarded to the OpenShift Routers. The Routers examine the HTTP headers of the queries and forward them to the correct destination.

With a load-balancer host address of 10.19.x.y, the wildcard DNS record can be added as follows:

Table 1.5. Load Balancer DNS records

IP Address	Hostname	Purpose
10.19.x.y	`*.apps.example.com`	User access to application web services

1.7. Bastion Instance

Best practices recommend minimizing attack vectors into a system by exposing only those services required by consumers of the system. In the event of failure or a need for manual configuration, systems administrators require further access to internal components in the form of secure administrative back-doors.

In the case of Red Hat OpenShift Container Platform running in a cloud provider context, the entry points to the Red Hat OpenShift Container Platform infrastructure such as the API, Web Console and routers are the only services exposed to the outside. The systems administrators' access from the public network space to the private network is possible with the use of a bastion instance.

A bastion instance is a non-OpenShift instance accessible from outside of the Red Hat OpenShift Container Platform environment, configured to allow remote access via secure shell (ssh). To remotely access an instance, the systems administrator first accesses the bastion instance, then "jumps" via another ssh connection to the intended OpenShift instance. The bastion instance may be referred to as a "jump host".

Note

As the bastion instance can access all internal instances, it is recommended to take extra measures to harden this instance’s security. For more information on hardening the bastion instance, see the official Guide to Securing Red Hat Enterprise Linux 7

Depending on the environment, the bastion instance may be an ideal candidate for running administrative tasks such as the Red Hat OpenShift Container Platform installation playbooks. This reference environment uses the bastion instance for the installation of the Red Hat OpenShift Container Platform.

1.8. Red Hat OpenShift Container Platform Components

Red Hat OpenShift Container Platform comprises of multiple instances running on Red Hat OpenStack Platform that allow for scheduled and configured OpenShift services and supplementary containers. These containers can have persistent storage, if required, by the application and integrate with optional OpenShift services such as logging and metrics.

1.8.1. OpenShift Instances

Instances running the Red Hat OpenShift Container Platform environment run the atomic-openshift-node service that allows for the container orchestration of scheduling pods. The following sections describe the different instance and their roles to develop a Red Hat OpenShift Container Platform solution.

1.8.1.1. Master Instances

Master instances run the OpenShift master components, including the API server, controller manager server, and optionally etcd. The master manages nodes in its Kubernetes cluster and schedules pods to run on nodes.

Note

The master instances are considered nodes as well and run the atomic-openshift-node service.

For optimal performance, the etcd service should run on the masters instances. When collocating etcd with master nodes, at least three instances are required. In order to have a single entry-point for the API, the master nodes should be deployed behind a load balancer.

In order to create master instances with labels, set the following in the inventory file as:

... [OUTPUT ABBREVIATED] ...
[etcd]
master1.example.com
master2.example.com
master3.example.com

[masters]
master1.example.com
master2.example.com
master3.example.com

[nodes]
master1.example.com openshift_node_labels="{'region': 'master', 'masterlabel2': 'value2'}"
master2.example.com openshift_node_labels="{'region': 'master', 'masterlabel2': 'value2'}"
master3.example.com openshift_node_labels="{'region': 'master', 'masterlabel2': 'value2'}"

Ensure the openshift_web_console_nodeselector ansible variable value matches with a master node label in the inventory file. By default, the web_console is deployed to the masters.

Note

See the official OpenShift documentation for a detailed explanation on master nodes.

1.8.1.2. Infrastructure Instances

The infrastructure instances run the atomic-openshift-node service and host the Red Hat OpenShift Container Platform components such as Registry, Prometheus and Hawkular metrics. The infrastructure instances also run the Elastic Search, Fluentd, and Kibana(EFK) containers for aggregate logging. Persistent storage should be available to the services running on these nodes.

Depending on environment requirements at least three infrastructure nodes are required to provide a sharded/highly available aggregated logging service and to ensure that service interruptions do not occur during a reboot.

Note

For more infrastructure considerations, visit the official OpenShift documentation.

When creating infrastructure instances with labels, set the following in the inventory file as:

... [OUTPUT ABBREVIATED] ...
[nodes]
infra1.example.com openshift_node_labels="{'region': 'infra', 'infralabel1': 'value1'}"
infra2.example.com openshift_node_labels="{'region': 'infra', 'infralabel1': 'value1'}"
infra3.example.com openshift_node_labels="{'region': 'infra', 'infralabel1': 'value1'}"

Note

The router and registry pods automatically are scheduled on nodes with the label of 'region': 'infra'.

1.8.1.3. Application Instances

The Application (app) instances run the atomic-openshift-node service. These nodes should be used to run containers created by the end users of the OpenShift service.

When creating node instances with labels, set the following in the inventory file as:

... [OUTPUT ABBREVIATED] ...

[nodes]
node1.example.com openshift_node_labels="{'region': 'primary', 'nodelabel2': 'value2'}"
node2.example.com openshift_node_labels="{'region': 'primary', 'nodelabel2': 'value2'}"
node3.example.com openshift_node_labels="{'region': 'primary', 'nodelabel2': 'value2'}"

1.8.2. etcd

etcd is a consistent and highly-available key value store used as Red Hat OpenShift Container Platform’s backing store for all cluster data. etcd stores the persistent master state while other components watch etcd for changes to bring themselves into the desired state.

Since values stored in etcd are critical to the function of Red Hat OpenShift Container Platform, firewalls should be implemented to limit the communication with etcd nodes. Inter-cluster and client-cluster communication is secured by utilizing x509 Public Key Infrastructure (PKI) key and certificate pairs.

etcd uses the RAFT algorithm to gracefully handle leader elections during network partitions and the loss of the current leader. For a highly available Red Hat OpenShift Container Platform deployment, an odd number (starting with three) of etcd instances are required.

1.8.3. Labels

Labels are key/value pairs attached to objects such as pods. They are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users but do not directly imply semantics to the core system. Labels can also be used to organize and select subsets of objects. Each object can have a set of labels defined at creation time or subsequently added and modified at any time.

Note

Each key must be unique for a given object.

"labels": {
  "key1" : "value1",
  "key2" : "value2"
}

Index and reverse-index labels are used for efficient queries, watches, sorting and grouping in UIs and CLIs, etc. Labels should not be polluted with non-identifying, large and/or structured data. Non-identifying information should instead be recorded using annotations.

1.8.3.1. Labels as Alternative Hierarchy

Service deployments and batch processing pipelines are often multi-dimensional entities (e.g., multiple partitions or deployments, multiple release tracks, multiple tiers, multiple micro-services per tier). Management of these deployments often requires cutting across the encapsulation of strictly hierarchical representations—especially those rigid hierarchies determined by the infrastructure rather than by users. Labels enable users to map their own organizational structures onto system objects in a loosely coupled fashion, without requiring clients to store these mappings.

Example labels:

{"release" : "stable", "release" : "canary"}
{"environment" : "dev", "environment" : "qa", "environment" : "production"}
{"tier" : "frontend", "tier" : "backend", "tier" : "cache"}
{"partition" : "customerA", "partition" : "customerB"}
{"track" : "daily", "track" : "weekly"}

These are just examples of commonly used labels; the ability exists to develop specific conventions that best suit the deployed environment.

1.8.3.2. Labels as Node Selector

Node labels can be used as node selector where different nodes can be labeled to different use cases. The typical use case is to have nodes running Red Hat OpenShift Container Platform infrastructure components like the Red Hat OpenShift Container Platform registry, routers, metrics or logging components named "infrastructure nodes" to differentiate them from nodes dedicated to run user applications. Following this use case, the admin can label the "infrastructure nodes" with the label "region=infra" and the application nodes as "region=app". Other uses can be having different hardware in the nodes and have classifications like "type=gold", "type=silver" or "type=bronze".

The scheduler can be configured to use node labels to assign pods to nodes depending on the node-selector. At times it makes sense to have different types of nodes to run certain pods, the node-selector can be set to select which labels are used to assign pods to nodes.

1.9. Software Defined Networking

Red Hat OpenShift Container Platform offers the ability to specify how pods communicate with each other. This could be through the use of Red Hat provided Software-defined networks (SDN), a third-party SDN, or flannel.

Deciding on the appropriate internal network for an Red Hat OpenShift Container Platform step is a crucial step. Unfortunately, there is no right answer regarding the appropriate pod network to chose, as this varies based upon the specific scenario requirements on how a Red Hat OpenShift Container Platform environment is to be used.

For the purposes of this reference environment, the Red Hat OpenShift Container Platform ovs-networkpolicy SDN plug-in is chosen due to its ability to provide pod isolation using Kubernetes NetworkPolicy. The following section, “OpenShift SDN Plugins and Flannel”, discusses pros and cons when deciding between the four popular options for the internal networks - ovs-multitenant, ovs-networkpolicy, ovs-subnet, and flannel.

1.9.1. OpenShift SDN Plugins and Flannel

This section focuses on two methods for pod communication within Red Hat OpenShift Container Platform, OpenShift SDN and flannel. Regarding OpenShift SDN, there are three plugin options are listed below.

ovs-subnet - the original plugin that provides an overlay network created to allow pod-to-pod communication and services. This pod network is created using Open vSwitch (OVS).
ovs-multitenant - a plugin that provides an overlay network that is configured using OVS, similar to the ovs-subnet plugin, however, unlike the ovs-subnet it provides Red Hat OpenShift Container Platform project level isolation for pods and services.
ovs-networkpolicy - a plugin that provides an overlay network that is configured using OVS that provides the ability for Red Hat OpenShift Container Platform administrators to configure specific isolation policies using NetworkPolicy objects².

2: https://docs.openshift.com/container-platform/3.9/admin_guide/managing_networking.html#admin-guide-networking-networkpolicy

When comparing the three different OpenShift SDN options, one common drawback that each possesses is the fact that they use an overlay network that uses OVS. The reason it is a drawback is specific to when deploying Red Hat OpenShift Container Platform on Red Hat OpenStack Platform as Red Hat OpenShift Container Platform internode traffic is carried over a Red Hat OpenStack Platform neutron network. neutron is a SDN that uses OVS and provides network access to all the Red Hat OpenShift Container Platform instances. This in turn causes double encapsulation and may degrade performance of the pod network.

What about flannel?

flannel is a virtual networking layer designed for containers. Instead of having two SDN to route traffic, flannel runs in a mode labeled host-gw that routes packets by each node in the Red Hat OpenShift Container Platform environment running a daemon labeled flanneld that sends information to a centralized location (etcd store) thus allowing other nodes running the flanneld daemon to route packets to other containers as long as they are all on the flannel network. The main benefit to flannel is that it provides performance benefits over OVS double encapsulation since it does not create its own SDN on top of the existing RHOSP neutron network.

Why not default to flannel?

flannel uses a single IP network space for all of the pods allocating a contiguous subset of the space to each instance. Consequently, nothing prevents a pod from attempting to contact any IP address in the same network space. This hinders multi-tenancy because the network cannot be used to isolate pods in one application from another.

The key to making the right decision rests between mutli-tenancy isolation vs performance. Depending on the priority of each of these, should determine the appropriate choice when deciding between OpenShift SDN and flannel internal networks.

The steps to setup flannel instead of OpenShift SDN can be found within the Appendix B, Using Flannel.

Network isolation is important, which OpenShift SDN to choose?

With the above, this leaves two OpenShift SDN options: ovs-multitenant and ovs-networkpolicy. The reason ovs-subnet is ruled out is due to it not having network isolation. If isolation is not important, flannel is a better choice since it would not have the double encapsulation drawback.

While both ovs-multitenant and ovs-networkpolicy provide network isolation, the optimal choice comes down to what type of isolation is required. The ovs-multitenant plugin provides project-level isolation for pods and services. This means that pods and services from different projects cannot communicate with each other.

On the other hand, ovs-networkpolicy solves network isolation by providing project administrators the flexibility to create their own network policies using Kubernetes NetworkPolicy objects. This means that by default all pods in a project are accessible from other pods and network endpoints until NetworkPolicy objects are created. This in turn may allow pods from separate projects to communicate with each other assuming the appropriate NetworkPolicy is in place.

Depending on the level of isolation required, should determine the appropriate choice when deciding between ovs-multitenant and ovs-networkpolicy.

1.10. Container Storage

Container images are stored locally on the nodes running Red Hat OpenShift Container Platform pods. The container-storage-setup script uses the /etc/sysconfig/docker-storage-setup file to specify the storage configuration.

The /etc/sysconfig/docker-storage-setup file should be created before starting the docker service, otherwise the storage would be configured using a loopback device. The container storage setup is performed on all hosts running containers, therefore masters, infrastructure, and application nodes.

1.11. Persistent Storage

Containers by default offer ephemeral storage but some applications require the storage to persist between different container deployments or upon container migration. Persistent Volume Claims (PVC) are used to store the application data. These claims can either be added into the environment by hand or provisioned dynamically using a StorageClass object.

1.11.1. Storage Classes

The StorageClass resource object describes and classifies different types of storage that can be requested, as well as provides a means for passing parameters to the backend for dynamically provisioned storage on demand. StorageClass objects can also serve as a management mechanism for controlling different levels of storage and access to the storage. Cluster Administrators (cluster-admin) or Storage Administrators (storage-admin) define and create the StorageClass objects that users can use without needing any intimate knowledge about the underlying storage volume sources. Because of this the naming of the storage class defined in the StorageClass object should be useful in understanding the type of storage it maps whether that is storage from Red Hat OpenStack Platform or from glusterfs if deployed.

1.11.1.1. Persistent Volumes

Persistent volumes (PV) provide pods with non-ephemeral storage by configuring and encapsulating underlying storage sources. A persistent volume claim (PVC) abstracts an underlying PV to provide provider agnostic storage to OpenShift resources. A PVC, when successfully fulfilled by the system, mounts the persistent storage to a specific directory (mountPath) within one or more pods. From the container point of view, the mountPath is connected to the underlying storage mount points by a bind-mount.

1.12. Registry

OpenShift can build containerimages from source code, deploy them, and manage their lifecycle. To enable this, OpenShift provides an internal, integrated registry that can be deployed in the OpenShift environment to manage images.

The registry stores images and metadata. For production environment, persistent storage should be used for the registry, otherwise any images that were built or pushed into the registry would disappear if the pod were to restart.

1.13. Aggregated Logging

One of the Red Hat OpenShift Container Platform optional components named Red Hat OpenShift Container Platform aggregated logging collects and aggregates logs from the pods running in the Red Hat OpenShift Container Platform cluster as well as /var/log/messages on nodes enabling Red Hat OpenShift Container Platform users to view the logs of projects which they have view access using a web interface.

Red Hat OpenShift Container Platform aggregated logging component it is a modified version of the ELK stack composed by a few pods running on the Red Hat OpenShift Container Platform environment:

Elasticsearch: An object store where all logs are stored.
Kibana: A web UI for Elasticsearch.
Curator: Elasticsearch maintenance operations performed automatically on a per-project basis.
Fluentd: Gathers logs from nodes and containers and feeds them to Elasticsearch.

Note

Fluentd can be configured to send a copy of the logs to a different log aggregator and/or to a different Elasticsearch cluster, see OpenShift documentation for more information.

Once deployed in the cluster, Fluentd (deployed as a DaemonSet on any node with the right labels) gathers logs from all nodes and containers, enriches the log document with useful metadata (e.g. namespace, container_name, node) and forwards them into Elasticsearch, where Kibana provides a web interface to users to be able to view any logs. Cluster administrators can view all logs, but application developers can only view logs for projects they have permission to view. To avoid users to see logs from pods in other projects, the Search Guard plugin for Elasticsearch is used.

A separate Elasticsearch cluster, a separate Kibana, and a separate Curator components can be deployed to form the OPS cluster where Fluentd send logs from the default, openshift, and openshift-infra projects as well as /var/log/messages on nodes into this different cluster. If the OPS cluster is not deployed those logs are hosted in the regular aggregated logging cluster.

Red Hat OpenShift Container Platform aggregated logging components can be customized for longer data persistence, pods limits, replicas of individual components, custom certificates, etc. The customization is provided by the Ansible variables as part of the deployment process.

The OPS cluster can be customized as well using the same variables using the suffix ops as in openshift_logging_es_ops_pvc_size.

Note

For more information about different customization parameters, see Aggregating Container Logs documentation.

Basic concepts for aggregated logging

Cluster: Set of Elasticsearch nodes distributing the workload
Node: Container running an instance of Elasticsearch, part of the cluster.
Index: Collection of documents (container logs)
Shards and Replicas: Indices can be split into sets of data containing the primary copy of the documents stored (primary shards) or backups of that primary copies (replica shards). Sharding allows the application to horizontally scaled the information and distributed/paralellized operations. Replication instead provides high availability and also better search throughput as searches are also executed on replicas.

Warning

Using NFS storage as a volume or a persistent volume (or via NAS such as Gluster) is not supported for Elasticsearch storage, as Lucene relies on file system behavior that NFS does not supply. Data corruption and other problems can occur.

By default every Elasticsearch pod of the Red Hat OpenShift Container Platform aggregated logging components has the role of Elasticsearch master and Elasticsearch data node. If only 2 Elasticsearch pods are deployed and one of the pods fails, all logging stops until the second master returns, so there is no availability advantage to deploy 2 Elasticsearch pods.

Note

Elasticsearch shards require their own storage, but Red Hat OpenShift Container Platform deploymentconfig shares storage volumes between all its pods, therefore every Elasticsearch pod is deployed using a different deploymentconfig so it cannot be scaled using oc scale. In order to scale the aggregated logging Elasticsearch replicas after the first deployment, it is required to modify the openshift_logging_es_cluser_size in the inventory file and re-run the openshift-logging.yml playbook.

Below is an example of some of the best practices when deploying Red Hat OpenShift Container Platform aggregated logging. Elasticsearch, Kibana, and Curator are deployed on nodes with the label of "region=infra". Specifying the node label ensures that the Elasticsearch and Kibana components are not competing with applications for resources. A highly-available environment for Elasticsearch is deployed to avoid data loss, therefore, at least 3 Elasticsearch replicas are deployed and openshift_logging_es_number_of_replicas parameter is configured to be 1 at least. The settings below would be defined in a variable file or static inventory.

openshift_logging_install_logging=true
openshift_logging_es_pvc_dynamic=true
openshift_logging_es_pvc_size=100Gi
openshift_logging_es_cluster_size=3
openshift_logging_es_nodeselector={"region":"infra"}
openshift_logging_kibana_nodeselector={"region":"infra"}
openshift_logging_curator_nodeselector={"region":"infra"}
openshift_logging_es_number_of_replicas=1

1.14. Aggregated Metrics

Red Hat OpenShift Container Platform has the ability to gather metrics from kubelet and store the values in Heapster. Red Hat OpenShift Container Platform Metrics provide the ability to view CPU, memory, and network-based metrics and display the values in the user interface. These metrics can allow for the horizontal autoscaling of pods based on parameters provided by an Red Hat OpenShift Container Platform user. It is important to understand capacity planning when deploying metrics into an Red Hat OpenShift Container Platform environment.

Red Hat OpenShift Container Platform metrics is composed by a few pods running on the Red Hat OpenShift Container Platform environment:

Heapster: Heapster scrapes the metrics for CPU, memory and network usage on every pod, then exports them into Hawkular Metrics.
Hawkular Metrics: A metrics engine that stores the data persistently in a Cassandra database.
Cassandra: Database where the metrics data is stored.

Red Hat OpenShift Container Platform metrics components can be customized for longer data persistence, pods limits, replicas of individual components, custom certificates, etc. The customization is provided by the Ansible variables as part of the deployment process.

As best practices when metrics are deployed, persistent storage should be used to allow for metrics to be preserved. Node selectors should be used to specify where the Metrics components should run. In the reference architecture environment, the components are deployed on nodes with the label of "region=infra".

openshift_metrics_install_metrics=True
openshift_metrics_storage_volume_size=20Gi
openshift_metrics_cassandra_storage_type=dynamic
openshift_metrics_hawkular_nodeselector={"region":"infra"}
openshift_metrics_cassandra_nodeselector={"region":"infra"}
openshift_metrics_heapster_nodeselector={"region":"infra"}

Where did the comment section go?

Red Hat's documentation publication system recently went through an upgrade to enable speedier, more mobile-friendly content. We decided to re-evaluate our commenting platform to ensure that it meets your expectations and serves as an optimal feedback mechanism. During this redesign, we invite your input on providing feedback on Red Hat documentation via the discussion platform.