Chapter 8. NBDE Tang Server Operator

8.1. NBDE Tang Server Operator overview

Network-bound Disk Encryption (NBDE) provides an automated unlocking of LUKS-encrypted volumes using one or more dedicated network-binding servers. The client side of NBDE is called the Clevis decryption policy framework and the server side is represented by Tang.

The NBDE Tang Server Operator allows the automation of deployments of one or several Tang servers in the OpenShift Container Platform (OCP) environment.

8.2. NBDE Tang Server Operator release notes

The following release notes track the development of the Security Profiles Operator in the OpenShift Container Platform.

8.3. Understanding the NBDE Tang Server Operator

You can use the NBDE Tang Server Operator to automate the deployment of a Tang server in an OpenShift Container Platform cluster that requires Network Bound Disk Encryption (NBDE) internally, leveraging the tools that OpenShift Container Platform provides to achieve this automation.

The NBDE Tang Server Operator simplifies the installation process and uses native features provided by the OpenShift Container Platform environment, such as multi-replica deployment, scaling, traffic load balancing, and so on. The Operator also provides automation of certain operations that are error-prone when you perform them manually, for example:

  • server deployment and configuration
  • key rotation
  • hidden keys deletion

The NBDE Tang Server Operator is implemented using the Operator SDK and allows the deployment of one or more Tang servers in OpenShift through custom resource definitions (CRDs).

8.3.1. Additional resources

8.4. Installing the NBDE Tang Server Operator

You can install the NBDE Tang Operator either by using the web console or through the oc command from CLI.

8.4.1. Installing the NBDE Tang Server Operator using the web console

You can install the NBDE Tang Server Operator from the OperatorHub using the web console.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.

Procedure

  1. In the OpenShift Container Platform web console, navigate to Operators → OperatorHub.

  2. Search for the NBDE Tang Server Operator:

    NBDE Tang Server Operator in OperatorHub
  3. Click Install.
  4. On the Operator Installation screen, keep the Update channel, Version, Installation mode, Installed Namespace, and Update approval fields on the default values.

  5. After you confirm the installation options by clicking Install, the console displays the installation confirmation.

    Confirmation of a NBDE Tang Server Operator installation

Verification

  1. Navigate to the Operators → Installed Operators page.

  2. Check that the NBDE Tang Server Operator is installed and its status is Succeeded.

    NBDE Tang Server Operator status

8.4.2. Installing the NBDE Tang Server Operator using CLI

You can install the NBDE Tang Server Operator from the OperatorHub using the CLI.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You have installed the OpenShift CLI (oc).

Procedure

  1. Use the following command to list available Operators on OperatorHub, and limit the output to Tang-related results: 

    $ oc get packagemanifests -n openshift-marketplace | grep tang

    Example output

    tang-operator           Red Hat

    In this case, the corresponding packagemanifest name is tang-operator.

  2. Create a Subscription object YAML file to subscribe a namespace to the NBDE Tang Server Operator, for example, tang-operator.yaml:

    Example subscription YAML for tang-operator

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: tang-operator
  namespace: openshift-operators
spec:
  channel: stable 1
  installPlanApproval: Automatic
  name: tang-operator 2
  source: redhat-operators 3
  sourceNamespace: openshift-marketplace 4

    1
    Specify the channel name from where you want to subscribe the Operator.
    2
    Specify the name of the Operator to subscribe to.
    3
    Specify the name of the CatalogSource that provides the Operator.
    4
    The namespace of the CatalogSource. Use openshift-marketplace for the default OperatorHub CatalogSources.

  3. Apply the Subscription to the cluster: 

    $ oc apply -f tang-operator.yaml

Verification

  • Check that the NBDE Tang Server Operator controller runs in the openshift-operators namespace: 

    $ oc -n openshift-operators get pods

    Example output

    NAME                                                READY   STATUS    RESTARTS   AGE
tang-operator-controller-manager-694b754bd6-4zk7x   2/2     Running   0          12s

8.5. Configuring and managing Tang servers using the NBDE Tang Server Operator

With the NBDE Tang Server Operator, you can deploy and quickly configure Tang servers. On the deployed Tang servers, you can list existing keys and rotate them.

8.5.1. Deploying a Tang server using the NBDE Tang Server Operator

You can deploy and quickly configure one or more Tang servers using the NBDE Tang Server Operator in the web console.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You have installed the NBDE Tang Server Operator on your OCP cluster.

Procedure

  1. In the OpenShift Container Platform web console, navigate to Operators → OperatorHub.

  2. Select Project, and click Create Project:

    Create Project in the web console

  3. On the Create Project page, fill in the required information, for example:

    Example values on the Create Project page
  4. Click Create.

  5. NBDE Tang Server replicas require a Persistent Volume Claim (PVC) for storing encryption keys. In the web console, navigate to Storage → PersistentVolumeClaims:

    PersistentVolumeClaims in the Storage menu
  6. On the following PersistentVolumeClaims screen, click Create PersistentVolumeClaim.

  7. On the Create PersistentVolumeClaim page, select a storage that fits your deployment scenario. Consider how often you want to rotate the encryption keys. Name your PVC and choose the claimed storage capacity, for example:

    Create PersistentVolumeClaims page
  8. Navigate to Operators → Installed Operators, and click NBDE Tang Server.

  9. Click Create instance.

    Create NBDE Tang Server instance

  10. On the Create TangServer page, choose the name of the Tang Server instance, amount of replicas, and specify the name of the previously created Persistent Volume Claim, for example:

    Create TangServer page
  11. After you enter the required values a change settings that differ from the default values in your scenario, click Create.

8.5.2. Rotating keys using the NBDE Tang Server Operator

With the NBDE Tang Server Operator, you also can rotate your Tang server keys. The precise interval at which you should rotate them depends on your application, key sizes, and institutional policy.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You deployed a Tang server using the NBDE Tang Server Operator on your OpenShift cluster.
  • You have installed the OpenShift CLI (oc).

Procedure

  1. List the existing keys on your Tang server, for example: 

    $ oc -n nbde describe tangserver

    Example output

    …
Status:
  Active Keys:
	File Name:  	QS82aXnPKA4XpfHr3umbA0r2iTbRcpWQ0VI2Qdhi6xg
	Generated:  	2022-02-08 15:44:17.030090484 +0000
	sha1:       	PvYQKtrTuYsMV2AomUeHrUWkCGg
	sha256:     	QS82aXnPKA4XpfHr3umbA0r2iTbRcpWQ0VI2Qdhi6xg
…

  2. Create a YAML file for moving your active keys to hidden keys, for example, minimal-keyretrieve-rotate-tangserver.yaml:

    Example key-rotation YAML for tang-operator

    apiVersion: daemons.redhat.com/v1alpha1
kind: TangServer
metadata:
  name: tangserver
  namespace: nbde
  finalizers:
    - finalizer.daemons.tangserver.redhat.com
spec:
  replicas: 1
  hiddenKeys:
    - sha1: "PvYQKtrTuYsMV2AomUeHrUWkCGg" 1

    1
    Specify the SHA-1 thumbprint of your active key to rotate it.

  3. Apply the YAML file: 

    $ oc apply -f minimal-keyretrieve-rotate-tangserver.yaml

Verification

  1. After a certain amount of time depending on your configuration, check that the previous activeKey value is the new hiddenKey value and the activeKey key file is newly generated, for example: 

    $ oc -n nbde describe tangserver

    Example output

    …
Spec:
  Hidden Keys:
    sha1:    PvYQKtrTuYsMV2AomUeHrUWkCGg
  Replicas:  1
Status:
  Active Keys:
    File Name:  T-0wx1HusMeWx4WMOk4eK97Q5u4dY5tamdDs7_ughnY.jwk
    Generated:  2023-10-25 15:38:18.134939752 +0000
    sha1:       vVxkNCNq7gygeeA9zrHrbc3_NZ4
    sha256:     T-0wx1HusMeWx4WMOk4eK97Q5u4dY5tamdDs7_ughnY
  Hidden Keys:
    File Name:           .QS82aXnPKA4XpfHr3umbA0r2iTbRcpWQ0VI2Qdhi6xg.jwk
    Generated:           2023-10-25 15:37:29.126928965 +0000
    Hidden:              2023-10-25 15:38:13.515467436 +0000
    sha1:                PvYQKtrTuYsMV2AomUeHrUWkCGg
    sha256:              QS82aXnPKA4XpfHr3umbA0r2iTbRcpWQ0VI2Qdhi6xg
…

8.5.3. Deleting hidden keys with the NBDE Tang Server Operator

After you rotate your Tang server keys, the previously active keys become hidden and are no longer advertised by the Tang instance. You can use the NBDE Tang Server Operator to remove encryption keys no longer used.

WARNING
Do not remove any hidden keys unless you are sure that all bound Clevis clients already use new keys.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You deployed a Tang server using the NBDE Tang Server Operator on your OpenShift cluster.
  • You have installed the OpenShift CLI (oc).

Procedure

  1. List the existing keys on your Tang server, for example: 

    $ oc -n nbde describe tangserver

    Example output

    …
Status:
  Active Keys:
	File Name:  	PvYQKtrTuYsMV2AomUeHrUWkCGg.jwk
	Generated:  	2022-02-08 15:44:17.030090484 +0000
	sha1:	    	PvYQKtrTuYsMV2AomUeHrUWkCGg
	sha256:	    	QS82aXnPKA4XpfHr3umbA0r2iTbRcpWQ0VI2Qdhi6xg
…

  2. Create a YAML file for removing all hidden keys, for example, hidden-keys-deletion-tangserver.yaml:

    Example hidden-keys-deletion YAML for tang-operator

    apiVersion: daemons.redhat.com/v1alpha1
kind: TangServer
metadata:
  name: tangserver
  namespace: nbde
  finalizers:
    - finalizer.daemons.tangserver.redhat.com
spec:
  replicas: 1
  hiddenKeys: [] 1

    1
    The empty array as the value of the hiddenKeys entry indicates you want to preserve no hidden keys on your Tang server.

  3. Apply the YAML file: 

    $ oc apply -f hidden-keys-deletion-tangserver.yaml

Verification

  1. After a certain amount of time depending on your configuration, check that the previous active key still exists, but no hidden key is available, for example: 

    $ oc -n nbde describe tangserver

    Example output

    …
Spec:
  Hidden Keys:
    sha1:    PvYQKtrTuYsMV2AomUeHrUWkCGg
  Replicas:  1
Status:
  Active Keys:
    File Name:  T-0wx1HusMeWx4WMOk4eK97Q5u4dY5tamdDs7_ughnY.jwk
    Generated:  2023-10-25 15:38:18.134939752 +0000
    sha1:       vVxkNCNq7gygeeA9zrHrbc3_NZ4
    sha256:     T-0wx1HusMeWx4WMOk4eK97Q5u4dY5tamdDs7_ughnY
Status:
  Ready:                 1
  Running:               1
  Service External URL:  http://35.222.247.84:7500/adv
  Tang Server Error:     No
Events:
…

8.6. Identifying URL of a Tang server deployed with the NBDE Tang Server Operator

Before you can configure your Clevis clients to use encryption keys advertised by your Tang servers, you must identify the URLs of the servers.

8.6.1. Identifying URL of the NBDE Tang Server Operator using the web console

You can identify the URLs of Tang servers deployed with the NBDE Tang Server Operator from the OperatorHub by using the OpenShift Container Platform web console. After you identify the URLs, you use the clevis luks bind command on your clients containing LUKS-encrypted volumes that you want to unlock automatically by using keys advertised by the Tang servers. See the Configuring manual enrollment of LUKS-encrypted volumes section in the RHEL 9 Security hardening document for detailed steps describing the configuration of clients with Clevis.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You deployed a Tang server by using the NBDE Tang Server Operator on your OpenShift cluster.

Procedure

  1. In the OpenShift Container Platform web console, navigate to Operators → Installed Operators → Tang Server.

  2. On the NBDE Tang Server Operator details page, select Tang Server.

    NBDE Tang Server Operator details
  3. The list of Tang servers deployed and available for your cluster appears. Click the name of the Tang server you want to bind with a Clevis client.

  4. The web console displays an overview of the selected Tang server. You can find the URL of your Tang server in the Tang Server External Url section of the screen:

    NBDE Tang Server Operator overview of a Tang server

    In this example, the URL of the Tang server is http://34.28.173.205:7500.

Verification

  • You can check that the Tang server is advertising by using curl, wget, or similar tools, for example: 

    $ curl 2> /dev/null http://34.28.173.205:7500/adv  | jq

    Example output

    {
  "payload": "eyJrZXlzIj…eSJdfV19",
  "protected": "eyJhbGciOiJFUzUxMiIsImN0eSI6Imp3ay1zZXQranNvbiJ9",
  "signature": "AUB0qSFx0FJLeTU…aV_GYWlDx50vCXKNyMMCRx"
}

8.6.2. Identifying URL of the NBDE Tang Server Operator using CLI

You can identify the URLs of Tang servers deployed with the NBDE Tang Server Operator from the OperatorHub by using the CLI. After you identify the URLs, you use the clevis luks bind command on your clients containing LUKS-encrypted volumes that you want to unlock automatically by using keys advertised by the Tang servers. See the Configuring manual enrollment of LUKS-encrypted volumes section in the RHEL 9 Security hardening document for detailed steps describing the configuration of clients with Clevis.

Prerequisites

  • You must have cluster-admin privileges on an OpenShift Container Platform cluster.
  • You have installed the OpenShift CLI (oc).
  • You deployed a Tang server by using the NBDE Tang Server Operator on your OpenShift cluster.

Procedure

  1. List details about your Tang server, for example: 

    $ oc -n nbde describe tangserver

    Example output

    …
Spec:
…
Status:
  Ready:                 1
  Running:               1
  Service External URL:  http://34.28.173.205:7500/adv
  Tang Server Error:     No
Events:
…

  2. Use the value of the Service External URL: item without the /adv part. In this example, the URL of the Tang server is http://34.28.173.205:7500.

Verification

  • You can check that the Tang server is advertising by using curl, wget, or similar tools, for example: 

    $ curl 2> /dev/null http://34.28.173.205:7500/adv  | jq

    Example output

    {
  "payload": "eyJrZXlzIj…eSJdfV19",
  "protected": "eyJhbGciOiJFUzUxMiIsImN0eSI6Imp3ay1zZXQranNvbiJ9",
  "signature": "AUB0qSFx0FJLeTU…aV_GYWlDx50vCXKNyMMCRx"
}

8.6.3. Additional resources