Heketi is a RESTful management interface which provides an API to manage GlusterFS nodes and volumes. Heketi users and admins can interact with this API, using the Go language, to manage the storage environment.

In situations where administrators have upgraded OpenShift Container Storage 3 following the exact steps posted in our Deployment Guide, any previously configured authentication parameters in the Heketi configuration are deleted during the update. This allows an unauthenticated user to perform actions with administrative privileges, including provisioning, maintaining, deleting Gluster storage, and the rest of the operations supported by the Heketi server API. Fresh installs of OpenShift Container Storage 3 are not affected.

Depending on the setup of OpenShift, the Heketi server API may be accessible from public networks as the API is available from wherever OpenShift is available. This means public access to the Heketi server API may be possible, based on the OpenShift configuration.

Solution

To fully resolve this issue, manual intervention is required by an OCS administrator.

The first step is to recover the admin key from the heketi-storage-admin-secret, which is the admin key that was previously set.

The following command should be executed in the namespace where the secret is located:

~]$ oc get secret heketi-storage-admin-secret -o jsonpath='{.data.key}'|base64 -d;echo

Use the admin key recovered in the previous step to configure heketi:

~]$ oc edit template heketi

Find the section which sets heketi user and heketi admin key. To mitigate this issue, please add ‘value’ in parameters and add your user and admin keys manually. The OCS admin can choose to set any phrase for the user key as long as it is not used by their infrastructure. It is not used by any of the OCS default installed resources. Set the admin key value to the admin key recovered from previous step.

...
- description: Set secret for those creating volumes as type _user_
  displayName: Heketi User Secret
  name: HEKETI_USER_KEY
  value: <your-user-key-here>

- description: Set secret for administration of the Heketi service as user _admin_
  displayName: Heketi Administrator Secret
  name: HEKETI_ADMIN_KEY
  value: <your-admin-key-here>

…

Once the Heketi template has been edited to include keys for the user and administrator, execute the steps below.

Get the name of heketi route,svc,deploymentconfigs and delete it:

~]$ oc get route,svc,dc -o wide
~]$ oc delete dc,svc,route heketi-${name}

After deleting, recreate the heketi process:

~]$ oc process heketi | oc create -f -

After this, the Heketi user and admin will be forced to authenticate first before executing any commands via the Heketi server API.

To check if Heketi is allowing users to run commands without authenticating run the following commands. The command should fail with the JWT error as shown if the authentication has been enabled.

~]$ unset HEKETI_CLI_USER HEKETI_CLI_KEY
~]$ heketi-cli cluster list
      Error: Invalid JWT token: Token missing iss claim

Note: The Deployment Guide has been corrected.