Compute node on which an application is running

or an application service.

Before containers are deployed, the goal of the container deployment (e.g. scaling, availability, disposable containers for security or CI/CD) MUST first be determined so that all security-related aspects of installation, operation and decommissioning can be planned.

This requirement must be implemented organizationally.

When planning, the operating costs that arise from container use or mixed operation SHOULD also be taken into account.

This requirement must be implemented organizationally.

The planning MUST be adequately documented.

This requirement must be implemented organizationally.

OpenShift supports all of the goals mentioned. Comprehensive handouts are available to carry out and document the planning of container use, security and compliance, architecture and installation on OpenShift. [SecGuide]

For containerized IT systems, it MUST be taken into account how containerization affects the IT systems and applications being operated, in particular the administration and suitability of the applications.

This requirement must be implemented organizationally.

Note: This requirement is actively supported by OpenShift. For example, OpenShift does not allow applications with fixed UID/GID settings as standard; on the contrary, these IDs are assigned dynamically (security-by-design). The behavior can be adjusted by administrators, for example for system tasks.

Based on the protection needs of the applications, it MUST be checked whether the requirements for isolation and encapsulation of the containerized IT systems and the virtual networks as well as the applications operated are sufficiently met.

This requirement must be implemented organizationally.

Note: OpenShift supports the requirements through strict client separation based on a “Project” (an extension to the Kubernetes namespace). The containers are separated from each other and from the host system via cgroups using SELinux. As long as all applications run “restricted” in the Security Context Constraint (SCC), OpenShift maintains strict client separation.

The operating system's own mechanisms SHOULD be included in this test.

This requirement must be implemented organizationally.

OpenShift supports this requirement by leveraging SELinux with its cgroups to create the container sandbox.

For virtual networks, the host performs the function of a network component. The building blocks of the sub-layers NET.1 networks and NET.3 network components MUST be taken into account accordingly.

This requirement must be implemented organizationally.

Logical and overlay networks MUST also be considered and modeled.

This requirement must be implemented organizationally.

Note: OpenShift supports different network infrastructures via the CNI plugin interface (e.g. OVN-Kubernetes, OpenShift-SDN, hardware networks etc.) OVN-Kubernetes, OpenShift-SDN, hardware networks etc.) The underlying network is abstracted by the network model in Openshift and usage is consistent across containers. This allows OpenShift to uniformly implement network security features such as: Firewall rules over network policies.

Furthermore, the containerized IT systems used MUST meet the requirements for availability and data throughput.

This requirement must be implemented organizationally.

Note: OpenShift provides fine-grained metrics for external capacity management via monitoring.

During ongoing operations, the performance and condition of the containerized IT systems SHOULD be monitored (so-called health checks).

OpenShift offers automated checks for the availability and health of an application. If the LivenessProbe (Health) repeatedly receives a negative result or is not reachable, the affected pod with the container is restarted. Using ReadinessProbe, a container can control whether it is ready to accept HTTP(S) based requests or not.

Note: Monitoring is considered in APP.4.4.A11.

The networks for the administration of the host, the administration of the containers and their access networks MUST be separated appropriately to the protection requirements.

Hosts and containers are controlled via the Kubernetes API. This is addressed via api.<cluster-fqdn>. The load balancer used for this is located in the administration network. The load balancer for *.apps.<cluster-fqdn> is set up separately in the active network. This means that the administration is appropriately separated.

The Console (the OpenShift web UI) is used by all users. Authorization takes place at the API level and is secured via RBAC.

The control plane is to be located in an administration network.

In principle, at least, administration of the host SHOULD only be possible from the administration network.

The web UI can be configured on another router that is terminated on the administration load balancer and is therefore only accessible from the administration network. This means that it can no longer be reached from the active network.

Only the communication relationships necessary for operation SHOULD be permitted.

This is a standard OpenShift feature. The OpenShift documentation [OpenShiftDocs] contains the necessary communication paths between control plane, infrastructure and worker nodes, as well as the necessary firewall activations of the underlying network stack at hardware or IaaS level. The communication between containers or pods within a client (“Project”) is not restricted by default, but can be regulated with micro-segmentation if necessary or as a service mesh with mTLS authentication be implemented (see APP.4.4.A18).

Externally exposed services can receive their own IP and thus data traffic can also be separated outside the platform. Inter-node communication is carried out via suitable tunnel protocols (VXLAN, GENEVE) and can also be encrypted using IPSec.

Storage of container logging data MUST occur outside of the container, at least on the container host.

OpenShift Logging stores the containers' logging data in a separate log management system. The output of the container is saved as long as it is on STDOUT and STDERR. The logging data can also be forwarded to different log storage depending on the source (e.g. infrastructure or application).

It is the responsibility of the applications to write log output to STDOUT and errors to STDERR.

The application or service that is to be operated in the container SHOULD be suitable for container operation.

This requirement must be implemented organizationally.

It SHOULD be taken into account that containers can more often terminate unexpectedly for the application running within them.

This requirement must be ensured as part of application development.

The results of the test according to SYS.1.6.A3 Secure use of containerized IT systems SHOULD be documented in a comprehensible manner.

This requirement must be implemented organizationally.

Suppliers must be contractually obliged to comply.

Each container SHOULD only provide one service at a time.

This requirement must be solved as part of application development.

ACS can check or enforce this rule using a policy.

Like software products, all images for production use SHOULD go through a testing and release process in accordance with module OPS.1.1.6 Software testing and releases.

This requirement must be solved organizationally.

Note: OpenShift offers various CI/CD solutions that can be used for automation. OpenShift Pipelines (Tekton-based) and traditional Jenkins are available directly in OpenShift. If the user uses gitlab-ci or github Actions , the runners can be executed in OpenShift. If the release process contains specific artifacts such as If you require SBOMs or the ability to statically analyze Dockerfiles, Quay and ACS can provide the necessary functionality.

For each container, resources on the host system, such as CPU, volatile and persistent memory, and network bandwidth, SHOULD be appropriately reserved and limited.

OpenShift supports the configuration of quotas for a project (client). Applications can have their resources appropriately limited using limits/requests.

Network bandwidth is limited at the pod level and can be determined separately according to incoming and outgoing network bandwidth. In addition, outgoing traffic (egress) can be marked at the namespace level with differentiated services code point (DSCP) classifications in order to assign quality of service classes to the outgoing packets in the physical network.

It SHOULD be defined and documented how the system reacts if these limitations are exceeded.

This requirement must be implemented organizationally.

Note: The behavior of OpenShift completely replicates the standard behavior of Kubernetes. If CPU limits are exceeded, the process is slowed down. If volatile memory is exceeded, the process is stopped and restarted by the scheduler. The persistent memory management is responsible for exceeding the persistent memory - OpenShift will not enforce or limit anything here. Compliance with the limited network bandwidth is enforced by dropping packets that exceed the limit.

The container runtime and all instantiated containers SHOULD only be run by a non-privileged system account that does not have or can obtain elevated rights to the container service or the host system's operating system.

With OpenShift, application containers run in the Security Context Constraint (SCC) “restricted” by default.

The container runtime SHOULD be encapsulated through additional measures, such as using CPU virtualization extensions.

OpenShift supports encapsulation by using SELinux. If necessary, entire nodes can also be encapsulated via underlying virtualization. This is always necessary when application containers require extended security context constraints (SCCs).

With the sandbox function based on Kata Containers, OpenShift provides a convenient way to isolate containers using virtualization technology.

If containers are to take over tasks of the host system in exceptional cases, the privileges on the host system SHOULD be limited to the necessary minimum.

OpenShift offers several SCC to restrict access to the network, file system or the host itself. This should only be allowed for administrative applications such as SIEM scanners or other infrastructure services that require access to the host. These SCCs should never be given to application containers.

Exceptions SHOULD be appropriately documented.

These exceptions must be documented in the operational documentation. A list of pods with the corresponding SCC annotation can serve as the basis for the documentation.

The containers SHOULD ONLY be able to access the mass storage and directories necessary for operation.

Applications can access persistent volumes (PVs) and temporary (ephemeral) storage in OpenShift. Persisted volumes are connected as network storage, ephemeral storage serves primarily as volatile, short-lived mass storage and is allocated within the container file system. This configures which PV can be reached and the use of the ephemeral storage is separated per pod. This means that each pod has its own volatile mass storage. Volumes can be limited in size.

Permissions SHOULD be granted explicitly only if they are needed.

OpenShift implements the principle of least privileges. The definition is made via an explicit configuration at the deployment level.

If the container runtime includes local storage for a container, the access rights in the file system SHOULD be restricted to the container's service account.

By default, no local storage is included. For reasons of reliability, this is explicitly not recommended.

If network storage is used, the permissions SHOULD be set on the network storage itself.

The network storage dictates the permissions. OpenShift supports this with the dynamically assigned UID/GID of the projects (clients).

Advanced policies SHOULD limit container permissions.

By default, OpenShift blocks the containers' permissions (security-by-default).

Mandatory Access Control (MAC) or comparable technology SHOULD enforce these policies.

OpenShift already uses SELinux Mandatory Access Control to restrict permissions by default Using the Security Profiles Operator [SecurityProfile], workload-dependent SELinux and Seccomp profiles can be created and managed.

Policies SHOULD restrict at least the following access:

• incoming and outgoing network connections,

• file system accesses and

• kernel requests (syscalls).

These permissions are managed in OpenShift and controlled via Security Context Constraints (SCCs). For tool-based policy management, ACS or Red Hat Advanced Cluster Management (ACM) (with Kyverno or Open Policy Agent) can be used.

The runtime SHOULD start the containers in such a way that the host system kernel prevents all activities of the containers that are not permitted by the policy (e.g. by setting up local packet filters or revoking permissions) or at least appropriately reports violations.

OpenShift already meets this requirement as standard (security-by-design).

Containers SHOULD not be able to change their file system at runtime.

This requirement must be implemented organizationally.

Note: By default, Red Hat recommends building containers so that the runtime UID does not have write permissions in the container. If the file system is changed (e.g. for a file system-based cache), this change will be lost when you restart, as the unchangeable image will be loaded again.

File systems SHOULD not be mounted with write permissions.

By default, local file systems are not mounted in containers. Containers access PVs that are integrated via OpenShift. This fulfills the requirement. Alternatively, ephemeral volumes can be used as volatile storage.

The container's root file system can be restricted to ReadOnly via the SecurityContext. Verification of this configuration can be carried out using ACS.

If containerized applications have high availability requirements, it SHOULD be decided at which level availability should be implemented (e.g. redundant at the host level).

OpenShift offers this by default (replicas and pod anti-affinities). The applications must support this high availability. Clusters can also be distributed across multiple fire zones (failure zones) within a region/location.

Before commissioning, it MUST be planned how the applications operated in the pods and their different test and production requirements
Operating environments are separated. Based on the protection needs of the applications, the planning MUST determine which architecture of namespaces, meta tags, clusters and networks adequately addresses the risks and whether virtualized servers and networks should also be used.

The planning MUST contain regulations for network, CPU and permanent memory separation. The separation SHOULD also take into account the network zone concept and the protection requirements and be tailored to these.

Applications SHOULD each run in their own Kubernetes namespace that includes all of the application's programs. Only applications with similar protection needs and similar potential attack vectors SHOULD share a Kubernetes cluster.

These requirements must be implemented organizationally. OpenShift fully supports them.

OpenShift simplifies the implementation of the stated requirements for separating applications as well as development and production environments by setting up projects (tenants). Namespaces, networks/network separation, meta tags as well as CPU and memory separation are already configured by OpenShift as required (security-by-design). Special requirements for protection and network zone concepts can also be flexibly and easily mapped using additional measures. This particularly includes the ability to define application classes, operate in multiple, separate clusters, and automatically distribute workloads to protection zones and fire compartments. Particularly in the case of separate clusters, ACM can support rule-based distribution of applications using labels.

Kubernetes and all other control plane applications MUST authenticate and authorize every action by a user or, in automated operation, corresponding software, regardless of whether the actions take place via a client, a web interface or via an appropriate interface (API). Administrative actions MUST NOT be carried out anonymously.

In the default configuration, OpenShift restricts the use of the web console and APIs only to authenticated and authorized users. Connection to external directory services (LDAP, OIDC and others) is possible.

Each user MUST ONLY receive the absolutely necessary rights. Permissions without restrictions MUST be granted very restrictively.

Only a small group of people SHOULD be allowed to define automation processes.

OpenShift already offers roles for a least privilege concept. The RBAC roles can be adapted or supplemented with new roles. The preconfigured roles enable easy authorization assignment according to the least-privilege and need-to-know principles. User actions can be tracked via the audit log.

Only selected administrators SHOULD be given the right to create or change persistent volume shares in Kubernetes.

In the default configuration, persistent storage can only be integrated by cluster administrators. For dynamically provisioned storage, the corresponding provisioners have the necessary authorizations. These provisioners must be set up and configured by an admin. Storage requirements are controlled and restricted using quota mechanisms.

The cluster MUST be backed up. Data backup MUST include:

• permanent storage (Persistent Volumes),

• configuration files from Kubernetes and other control plane programs,

• the current state of the Kubernetes cluster including the extensions,

• configuration databases, specifically here etcd,

• all infrastructure applications that are necessary to operate the cluster and the services within it and

• the data storage of the code and image registries.

Snapshots for the operation of the applications SHOULD also be considered. Snapshots MUST NOT replace data backup.

The data backup of a cluster must be individually defined as part of the system architecture as part of the operating model. The areas of responsibility for the container platform (cluster administration), the infrastructure services (system administration) and the application management (technical administration) should be considered separately.

For data backup as part of cluster administration (Kubernetes configuration, current state of the Kubernetes cluster, configuration database) the integrated functions or methods of OpenShift must be used. System administration and specialist administration must be carried out in accordance with the respective specifications.

Snapshots for persistent volumes are supported when using OpenShift's Container Storage Interface (CSI) drivers. OpenShift offers an easily configurable backup system with the OpenShift API for Data Protection (OADP).

Additional third-party solutions for backup are also available in the OperatorHub.

The networks for the administration of the nodes, the control plane and the individual networks of the application services SHOULD be separated.

ONLY the network ports of the pods necessary for operation SHOULD be released into the intended networks. For multiple applications on a Kubernetes cluster, all network connections between the Kubernetes namespaces SHOULD initially be prohibited and only required network connections should be permitted (whitelisting). The network ports necessary for the administration of the nodes, the runtime and Kubernetes including its extensions SHOULD ONLY be accessible from the administration network and from pods that require them.

Only selected administrators SHOULD have permission in Kubernetes to manage the CNI and create or change rules for the network.

The requirements for restricting network ports and network connections between Kubernetes namespaces are already supported by OpenShift as standard using network policies and the option for default network policies (security by design).

The separation of the management network can also be implemented at the namespace level via network policies (incoming, the responsibility of the namespace administrator) and egress firewalls (outgoing, the responsibility of the cluster admins).

Externally exposed services can receive their own IP and thus data traffic can also be separated outside the platform. Inter-node communication is carried out via suitable tunnel protocols (VXLAN, GENEVE) and can also be encrypted using IPSec.

The determination of the necessary network policies for applications is supported by the network policy generator in ACS.

Pods SHOULD NOT use the "default" service account. No rights SHOULD be granted to the “default” service account. Pods for different applications SHOULD each run under their own service accounts. Permissions for the service accounts of the application pods SHOULD be limited to those strictly necessary.

Pods that do not require a service account SHOULD not be able to view it or have access to corresponding tokens.

Only control plane pods and pods that absolutely need them SHOULD use privileged service accounts.

Automation programs SHOULD each receive their own tokens, even if they use a common service account due to similar tasks.

OpenShift has the necessary configuration options for this. On the OpenShift platform, the system namespaces (control plane and other system components) are generally only accessible to administrative users. System-side components also run with privileged permissions if necessary.

To implement the further requirements, please note the following when configuring the deployment:

• Creating a specific service account for deployment

• Definition of an individual role local to the namespace for deployment and assignment of the role to the service account

• This role should be defined to the minimum amount of permissions in accordance with the least privilege principle

• Analogous definition of service accounts, roles and role bindings for automation purposes

• Use Red Hat Advanced Cluster Security for Kubernetes policies to prevent use of the "default" service account.

In principle, security context constraints [SCC], as a further element of authorization control similar to the RBAC model, can be used by the platform administration to provide extended privileges in a fine-grained manner for those components that require such extended authorizations. This is also possible according to the least privilege principle.

Pods should only include the ServiceAccountToken if they actually use it. Otherwise, pods should deactivate automatic mounting via “automountServiceAccountToken=false”.

In pods, each container SHOULD define a health check for startup and operation (“readiness” and “liveness”). These checks SHOULD provide information about the availability of the software running in the pod. The checks SHOULD fail if the monitored software cannot perform its tasks properly. Each of these controls SHOULD define a time period appropriate to the service operating in the pod. Based on these checks, Kubernetes SHOULD delete or restart the pods.

These requirements must be implemented organizationally.

It must be implemented during application development and deployment configuration of the probes. OpenShift already provides the necessary functions for automatically stopping or deleting the pods and restarting them. The use of this functionality can be enforced using ACS.

There SHOULD be an automatic audit of the settings of the nodes, Kubernetes and the application pods against a defined list of permitted settings and against standardized benchmarks.

OpenShift provides an audit log file for all actions carried out. The audit configuration of the Openshift API server and the nodes can be done centrally from ACM on a policy-based basis when using multiple clusters. Alternatively, the audit settings should be configured and activated for each cluster.

Red Hat Advanced Cluster Security for Kubernetes (ACS) can check all managed resources against standardized and customized benchmarks. Violations are reported via OpenShift monitoring and documented in the violation log in ACS. Some of the benchmarks are included, can be obtained from the community and supplemented with your own definitions.

In addition, the Compliance Operator is a tool available that can automatically check the settings of the Openshift cluster against a defined profile at configurable time intervals. This profile can be one of the profiles supplied (e.g. Essentials 8, Center for Internet Security and others) or a profile tailored to your own needs.

Kubernetes SHOULD enforce the established rules in the cluster by connecting suitable tools.

Using Red Hat Advanced Cluster Management for Kubernetes (ACM), policies can be created for all managed resources, which are then enforced when the resources are created.

Applications with very high protection requirements SHOULD use their own Kubernetes clusters or dedicated nodes that are not available for other applications.

This requirement must be implemented organizationally. OpenShift supports implementation and enforcement reproducibly via multi-cluster management (Red Hat Advanced Cluster Management for Kubernetes) and the use of labels (see APP.4.4.A14).

Nodes SHOULD send a secured status message to the control plane, verified cryptographically and, if possible, with a TPM.

RHCOS, an immutable operating system, is used on the nodes. The operating system is specified by the control plane and the version is compared cryptographically (hash checksums) against the defined status. The configuration of the nodes is managed centrally by the control plane using MachineConfigs. Corresponding changes to the managed files are overwritten by the control plane.

The nodes are compared with the configuration stored in the control plane via cryptographically secured communication.

Using the file integrity operator provided by Red Hat, the files on the RHCOS nodes can be checked cryptographically using the Advanced Intrusion Detection Environment (AIDE) and changes to files can be detected [FileIntegrity]. The operator and its configuration can be managed using ACM.

OpenShift uses its own internal Certificate Authority (CA). The nodes communicate with the control plane via TLS connections (TLS v1.3), which are secured with node-specific certificates from this CA. The internal CA is created during the installation (bootstrap) process and managed in the control plane .

The control plane SHOULD ONLY include nodes in the cluster that have successfully proven their integrity.

As described, the control plane compares the nodes with the stored configuration. If a node is to be included, it establishes a connection to the control plane. The control plane enforces the central configuration. Only when the configuration phase is completed is the node released for workloads and therefore active.

All nodes must authenticate themselves using a certificate. This is transferred from the control plane during provisioning and is only valid for a very short time. Optionally, a cluster admin can be required to approve each request.

The operation SHOULD be structured in such a way that if one location fails, the clusters and thus the applications in the pods either continue to run without interruption or can restart at another location within a short period of time.

For the restart, all necessary configuration files, images, user data, network connections and other resources required for operation, including the hardware required for operation, SHOULD already be available at this location.

For the uninterrupted operation of the cluster, the Kubernetes control plane, the infrastructure applications of the cluster and the application pods SHOULD be distributed across several fire compartments based on location data from the nodes in such a way that the failure of one fire compartment does not lead to the failure of the application.

OpenShift supports topology labels to separate multiple fire compartments (so-called “failure zones”). Setting up complete fail-over clusters is also supported by Red Hat Advanced Cluster Management for Kubernetes in order to distribute the applications to the available clusters. With its configuration as a metro cluster, OpenShift Data Foundation offers the basis for an appropriate storage design that must be included in the development and deployment of applications.

Failures of individual nodes or entire fire compartments do not lead to a cluster or application failure as long as the applications and infrastructure services are distributed accordingly. This distribution must be carried out when deploying the application and infrastructure services using affinity rules.

In addition, Openshift Monitoring generates alarms if there are too few resources available for the control plane due to the failure of a node. Such alarms are also configurable for workloads.

If there is an increased risk of external influences and a very high need for protection, pods SHOULD be stopped and restarted regularly. No pod SHOULD run for more than 24 hours. The availability of the applications in the pod SHOULD be ensured.

A regular restart of pods can be forced using the Openshift Descheduler [Descheduler]. This makes it possible to automatically deprovision pods after a configurable lifetime (default 24 hours).

OpenShift provides sufficient mechanisms (e.g. rollout strategies, reconciliation loops, replica count) to ensure the availability of applications even after a forced restart. For non-cloud-native applications, consideration is required in terms of long-running processes and forced restarts of pods.