Modify Nutanix configuration of OCP cluster that was installed using the Assisted Installer

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (OCP) 4.12 and above
  • Nutanix Container Storage Interface (CSI) Driver

Issue

  • If a cluster is installed using the Assisted Installer with platform integration enabled, the Nutanix configuration needs to be updated manually on the installed cluster. This has to be done only once the installation is finished and the cluster is connected to console.redhat.com.

Note: This document does not apply to any other installation methods of the OpenShift cluster.

Resolution

After installing the OpenShift Container Platform on the Nutanix platform with platform integration enabled, you must update the following Nutanix configuration settings manually:

  • PRISMCENTRAL_ADDRESS - The Nutanix PrismCentral address
  • PRISMCENTRAL_PORT - The Nutanix PrismCentral port
  • PRISMELEMENT_ADDRESS - The Nutanix Prism Element address
  • PRISMELEMENT_PORT - The Nutanix Prism Element port
  • PRISMELEMENT_CLUSTERNAME - The Nutanix Prism Element cluster name
  • NUTANIX_USERNAME - The Nutanix Prism Element login
  • NUTANIX_PASSWORD - The Nutanix Prism Element password
  • NUTANIX_STORAGE_CONTAINER - The Nutanix Prism storage container
  1. In the OpenShift Container Platform command line interface, update the Nutanix configuration settings:
$ oc patch infrastructure/cluster --type=merge --patch-file=/dev/stdin <<-EOF
{
  "spec": {
    "platformSpec": {
      "nutanix": {
        "prismCentral": {
          "address": "${PRISMCENTRAL_ADDRESS}",
          "port": ${PRISMCENTRAL_PORT}
        },
        "prismElements": [
          {
            "endpoint": {
              "address": "${PRISMELEMENT_ADDRESS}",
              "port": ${PRISMELEMENT_PORT}
            },
            "name": "${PRISMELEMENT_CLUSTERNAME}"
          }
        ]
      },
      "type": "Nutanix"
    }
  }
}
EOF

infrastructure.config.openshift.io/cluster patched
  1. (Optional) In order to use MachineSets to create workers via Machine API see documentation and update the secret:
$ cat <<EOF | oc create -f -
apiVersion: v1
kind: Secret
metadata:
   name: nutanix-credentials
   namespace: openshift-cloud-controller-manager
type: Opaque
stringData:
  credentials: |
    [{"type":"basic_auth","data":{"prismCentral":{"username":"${NUTANIX_USERNAME}","password":"${NUTANIX_PASSWORD}"},"prismElements":null}}]
EOF

secret/nutanix-credentials created
  1. Update the Nutanix cloud provider configuration, if you are installing OpenShift Container Platform version 4.13 or later:
$ cat <<EOF | oc apply -f -
kind: ConfigMap
apiVersion: v1
metadata:
  name: cloud-provider-config
  namespace: openshift-config
data:
  config: |
    {
        "prismCentral": {
            "address": "${PRISMCENTRAL_ADDRESS}",
            "port":${PRISMCENTRAL_PORT},
            "credentialRef": {
                "kind": "Secret",
                "name": "nutanix-credentials",
                "namespace": "openshift-cloud-controller-manager"
            }
        },
        "topologyDiscovery": {
            "type": "Prism",
            "topologyCategories": null
        },
        "enableCustomLabeling": true
    }
EOF

Warning: resource configmaps/cloud-provider-config is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically.
configmap/cloud-provider-config configured
  1. Install the Nutanix CSI Operator:

    a. In the OpenShift Container Platform web console, navigate to OperatorsOperatorHub.
    b. Search for the keyword "Nutanix" and select Nutanix CSI Operator. Then click Install.
    c. On the Install Operator page, leave the default settings and click Install.

    For more information, see Nutanix documentation.

  2. Create a NutanixCsiStorage resource to deploy your driver. You can do it directly inside the Operator UI with the Create instance button or with the following resource:

$ cat <<EOF | oc create -f -
apiVersion: crd.nutanix.com/v1alpha1
kind: NutanixCsiStorage
metadata:
  name: nutanixcsistorage
  namespace: openshift-cluster-csi-drivers
spec: {}
EOF
  1. Create a Nutanix secret for the CSI storage driver:
$ cat <<EOF | oc create -f -
apiVersion: v1
kind: Secret
metadata:
  name: ntnx-secret
  namespace: openshift-cluster-csi-drivers
stringData:
  # prism-element-ip:prism-port:admin:password
  key: PRISMELEMENT_ADDRESS:PRISMELEMENT_PORT:NUTANIX_USERNAME:NUTANIX_PASSWORD
EOF

secret/ntnx-secret created

Diagnostic Steps

Steps to verify Nutanix in Openshift:

  1. Create a storage class
$ cat <<EOF | oc create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nutanix-volume
  annotations:
    storageclass.kubernetes.io/is-default-class: 'true'
provisioner: csi.nutanix.com
parameters:
  csi.storage.k8s.io/fstype: ext4
  csi.storage.k8s.io/provisioner-secret-namespace: openshift-cluster-csi-drivers
  csi.storage.k8s.io/provisioner-secret-name: ntnx-secret
  storageContainer: NUTANIX_STORAGE_CONTAINER
  csi.storage.k8s.io/controller-expand-secret-name: ntnx-secret
  csi.storage.k8s.io/node-publish-secret-namespace: openshift-cluster-csi-drivers
  storageType: NutanixVolumes
  csi.storage.k8s.io/node-publish-secret-name: ntnx-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: openshift-cluster-csi-drivers
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
EOF
  1. Verify that you can create the Nutanix persistent volume claim (PVC)
    a. Create the persistent volume claim (PVC):
$ cat <<EOF | oc create -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nutanix-volume-pvc
  namespace: openshift-cluster-csi-drivers
  annotations:
    volume.beta.kubernetes.io/storage-provisioner: csi.nutanix.com
    volume.kubernetes.io/storage-provisioner: csi.nutanix.com
  finalizers:
    - kubernetes.io/pvc-protection
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: nutanix-volume
  volumeMode: Filesystem
EOF

persistentvolumeclaim/nutanix-volume-pvc created

b. Validate the persistent volume claim (PVC) status is Bound:

$ oc get pvc -n openshift-cluster-csi-drivers
NAME                 STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS     AGE
nutanix-volume-pvc   Bound                                      nutanix-volume   52s

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments