Chapter 3. Deploying AMQ Broker on OpenShift Container Platform using the AMQ Broker Operator

3.1. Installing the Operator using the CLI

The procedures in this section show how to use the OpenShift command-line interface (CLI) to install and deploy the latest version of the Operator for AMQ Broker 7.7 in a given OpenShift project. In subsequent procedures, you use this Operator to deploy some broker instances.

For an alternative method of installing the AMQ Broker Operator that uses the OperatorHub graphical interface, see Section 3.2, “Installing the Operator using the Operator Lifecycle Manager”.

Important
  • Deploying the Custom Resource Definitions (CRDs) that accompany the AMQ Broker Operator requires cluster administrator privileges for your OpenShift cluster. When the Operator is deployed, non-administrator users can create broker instances via corresponding Custom Resources (CRs). To enable regular users to deploy CRs, the cluster administrator must first assign roles and permissions to the CRDs. For more information, see Creating cluster roles for Custom Resource Definitions in the OpenShift Container Platform documentation.
  • If you previously deployed:

    • Version 0.15 of the Operator (that is, the first version available for AMQ Broker 7.7) you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.1, “Upgrading version 0.15 of the Operator”.
    • Version 0.13 of the Operator (that is, the version available for AMQ Broker 7.6) you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.2, “Upgrading version 0.13 of the Operator”.
    • Version 0.9 of the Operator (that is, the version available for AMQ Broker 7.5 or the Long Term Suppport (LTS) version available for AMQ Broker 7.4), you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.3, “Upgrading version 0.9 of the Operator”.
  • When you update your cluster with the CRDs for the latest Operator version, this update affects all projects in the cluster. Any broker Pods deployed from previous versions of the Operator might become unable to update their status. When you click the Logs tab of a running broker Pod in the OpenShift Container Platform web console, you see messages indicating that 'UpdatePodStatus' has failed. However, the broker Pods and Operator in that project continue to work as expected. To fix this issue for an affected project, you must also upgrade that project to use the latest version of the Operator.

3.1.1. Getting the Operator code

This procedure shows how to access and prepare the code you need to install the latest version of the Operator for AMQ Broker 7.7.

Procedure

  1. In your web browser, navigate to the Software Downloads page for AMQ Broker 7.7.0 patches.
  2. Ensure that the value of the Version drop-down list is set to 7.7.0 and the Patches tab is selected.
  3. Next to AMQ Broker 7.7.0 Operator Installation and Example Files, click Download.

    Download of the amq-broker-operator-7.7.0-ocp-install-examples.zip compressed archive automatically begins.

  4. When the download has completed, move the archive to your chosen installation directory. The following example moves the archive to a directory called ~/broker/operator.

    mkdir ~/broker/operator
    mv amq-broker-operator-7.7.0-ocp-install-examples.zip ~/broker/operator
  5. In your chosen installation directory, extract the contents of the archive. For example:

    cd ~/broker/operator
    unzip amq-broker-operator-7.7.0-ocp-install-examples.zip
  6. Log in to OpenShift Container Platform as a cluster administrator. For example:

    $ oc login -u system:admin
  7. Specify the project in which you want to install the Operator. You can create a new project or switch to an existing one.

    1. Create a new project:

      $ oc new-project <project_name>
    2. Or, switch to an existing project:

      $ oc project <project_name>
  8. Specify a service account to use with the Operator.

    1. In the deploy directory of the Operator archive that you extracted, open the service_account.yaml file.
    2. Ensure that the kind element is set to ServiceAccount.
    3. In the metadata section, assign a custom name to the service account, or use the default name. The default name is amq-broker-operator.
    4. Create the service account in your project.

      $ oc create -f deploy/service_account.yaml
  9. Specify a role name for the Operator.

    1. Open the role.yaml file. This file specifies the resources that the Operator can use and modify.
    2. Ensure that the kind element is set to Role.
    3. In the metadata section, assign a custom name to the role, or use the default name. The default name is amq-broker-operator.
    4. Create the role in your project.

      $ oc create -f deploy/role.yaml
  10. Specify a role binding for the Operator. The role binding binds the previously-created service account to the Operator role, based on the names you specified.

    1. Open the role_binding.yaml file. Ensure that the name values for ServiceAccount and Role match those specified in the service_account.yaml and role.yaml files. For example:

      metadata:
          name: amq-broker-operator
      subjects:
          kind: ServiceAccount
          name: amq-broker-operator
      roleRef:
          kind: Role
          name: amq-broker-operator
    2. Create the role binding in your project.

      $ oc create -f deploy/role_binding.yaml

3.1.2. Deploying the Operator using the CLI

The procedure in this section shows how to use the OpenShift command-line interface (CLI) to deploy the latest version of the Operator for AMQ Broker 7.7 in your OpenShift project.

Important

If you previously deployed:

  • Version 0.15 of the Operator (that is, the first version available for AMQ Broker 7.7) you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.1, “Upgrading version 0.15 of the Operator”.
  • Version 0.13 of the Operator (that is, the version available for AMQ Broker 7.6) you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.2, “Upgrading version 0.13 of the Operator”.
  • Version 0.9 of the Operator (that is, the version available for AMQ Broker 7.5 or the Long Term Suppport (LTS) version available for AMQ Broker 7.4), you can upgrade your project to use the latest version of the Operator, rather than performing a new installation. When you upgrade, you do not need to delete any existing CRDs in your OpenShift cluster. However, as part of the upgrade, you must delete the main broker Custom Resource (CR) instance from your project. Then, when you have upgraded the Operator, you must deploy a new CR to recreate your broker deployment. For more information, see Section 5.2.3, “Upgrading version 0.9 of the Operator”.
  • Version 0.6 of the Operator (that is, the Technical Preview version for AMQ Broker 7.4.0) you cannot directly upgrade this Operator to later versions, including the latest version available for AMQ Broker 7.7. The Custom Resource Definitions (CRDs) used by version 0.6 are not compatible with later versions of the Operator. Instead, you must perform a new installation of the Operator. This involves deleting any existing Custom Resource Definitions (CRDs) installed in your OpenShift cluster. These steps are described in the procedure in this section.

Prerequisites

  • You have already prepared your OpenShift project for the Operator deployment. See Section 3.1.1, “Getting the Operator code”.
  • Starting in AMQ Broker 7.3, you use a new version of the Red Hat Container Registry to access container images. This new version of the registry requires you to become an authenticated user before you can access images. Before you can follow the procedure in this section, you must first complete the steps described in Red Hat Container Registry Authentication.
  • If you intend to deploy brokers with persistent storage and do not have container-native storage in your OpenShift cluster, you need to manually provision Persistent Volumes (PVs) and ensure that they are available to be claimed by the Operator. For example, if you want to create a cluster of two brokers with persistent storage (that is, by setting persistenceEnabled=true in your Custom Resource), you need to have two PVs available. By default, each broker instance requires storage of 2 GiB.

    If you specify persistenceEnabled=false in your Custom Resource, the deployed brokers uses ephemeral storage. Ephemeral storage means that that every time you restart the broker Pods, any existing data is lost.

    For more information about provisioning persistent storage, see:

  • To learn how to specify the size of Persistent Volume Claim (PVC) that each broker in your deployment requires for persistent message storage, see Section 4.2.1, “Configuring broker storage size”.

Procedure

  1. In the OpenShift Container Platform web console, open the project in which you want your broker deployment.

    If you created a new project, it is currently empty. Observe that there are no Deployments, StatefulSets, Pods, Services, or Routes.

  2. If you deployed an earlier version of the AMQ Broker Operator in the project, remove the main broker Custom Resource (CR) that you used to create your broker deployment. Deleting the main CR removes the existing broker deployment in the project. For example:

    oc delete -f deploy/crs/broker_v1alpha1_activemqartemis_cr.yaml.
  3. If you deployed an earlier version of the AMQ Broker Operator in the project, delete this Operator instance. For example:

    $ oc delete -f deploy/operator.yaml
  4. If you deployed Custom Resource Definitions (CRDs) in your OpenShift cluster for an earlier version of the AMQ Broker Operator, remove these CRDs from the cluster. For example:

    oc delete -f deploy/crds/broker_v1alpha1_activemqartemis_crd.yaml
    oc delete -f deploy/crds/broker_v1alpha1_activemqartemisaddress_crd.yaml
    oc delete -f deploy/crds/broker_v1alpha1_activemqartemisscaledown_crd.yaml
  5. Deploy the CRDs that are included in the deploy/crds directory of the Operator archive that you downloaded and extracted. You must install the latest CRDs in your OpenShift cluster before deploying and starting the Operator.

    1. Deploy the main broker CRD.

      $ oc create -f deploy/crds/broker_activemqartemis_crd.yaml
    2. Deploy the address CRD.

      $ oc create -f deploy/crds/broker_activemqartemisaddress_crd.yaml
    3. Deploy the scaledown controller CRD.

      $ oc create -f deploy/crds/broker_activemqartemisscaledown_crd.yaml
  6. Link the pull secret associated with the account used for authentication in the Red Hat Container Registry with the default, deployer, and builder service accounts for your OpenShift project.

    $ oc secrets link --for=pull default <secret-name>
    $ oc secrets link --for=pull deployer <secret-name>
    $ oc secrets link --for=pull builder <secret-name>
    Note

    In OpenShift Container Platform 4.1 or later, you can also use the web console to associate a pull secret with a project in which you want to deploy container images such as the AMQ Broker Operator. To do this, click AdministrationService Accounts. Specify the pull secret associated with the account that you use for authentication in the Red Hat Container Registry.

  7. In the deploy directory of the Operator archive that you downloaded and extracted, open the operator.yaml file. Ensure that the value of the spec.containers.image property is set to the latest Operator image for AMQ Broker 7.7, as shown below.

    spec:
        template:
            spec:
                containers:
                    image: registry.redhat.io/amq7/amq-broker-rhel7-operator:0.17
  8. Deploy the Operator.

    $ oc create -f deploy/operator.yaml

    In your OpenShift project, the amq-broker-operator image that you deployed starts in a new Pod.

    The information on the Events tab of the new Pod confirms that OpenShift has deployed the Operator image you specified, assigned a new container to a node in your OpenShift cluster, and started the new container.

    In addition, if you click the Logs tab within the Pod, the output should include lines resembling the following:

    ...
    {"level":"info","ts":1553619035.8302743,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"activemqartemisaddress-controller"}
    {"level":"info","ts":1553619035.830541,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"activemqartemis-controller"}
    {"level":"info","ts":1553619035.9306898,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"activemqartemisaddress-controller","worker count":1}
    {"level":"info","ts":1553619035.9311671,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"activemqartemis-controller","worker count":1}

    The preceding output confirms that the newly-deployed Operator is communicating with Kubernetes, that the controllers for the broker and addressing are running, and that these controllers have started some workers.

Note

It is recommended that you deploy only a single instance of the AMQ Broker Operator in a given OpenShift project. Specifically, setting the replicas element of your Operator deployment to a value greater than 1, or deploying the Operator more than once in the same project is not recommended.

Additional resources

3.2. Installing the Operator using the Operator Lifecycle Manager

3.2.1. Overview of the Operator Lifecycle Manager

In OpenShift Container Platform 4.1 and later, the Operator Lifecycle Manager (OLM) helps users install, update, and generally manage the lifecycle of all Operators and their associated services running across their clusters. It is part of the Operator Framework, an open source toolkit designed to manage Kubernetes native applications (Operators) in an effective, automated, and scalable way.

The OLM runs by default in OpenShift Container Platform 4.1 and later, which aids cluster administrators in installing, upgrading, and granting access to Operators running on their cluster. The OpenShift Container Platform web console provides management screens for cluster administrators to install Operators, as well as grant specific projects access to use the catalog of Operators available on the cluster.

OperatorHub is the graphical interface that OpenShift cluster administrators use to discover, install, and upgrade Operators. With one click, these Operators can be pulled from OperatorHub, installed on the cluster, and managed by the OLM, ready for engineering teams to self-service manage the software in development, test, and production environments.

When you have deployed the Operator, you can use Custom Resource (CR) instances to create broker deployments such as standalone and clustered brokers.

3.2.2. Installing the Operator in OperatorHub

In OperatorHub, the name of the Operator for AMQ Broker 7.7 is Red Hat Integration - AMQ Broker. If you do not see the Operator automatically available in OperatorHub, follow this procedure to manually install the Operator in OperatorHub.

Procedure

  1. In your web browser, navigate to the AMQ Broker Software Downloads page.
  2. In the Version drop-down box, ensure the value is set to the latest AMQ Broker version, 7.7.0.
  3. Click the Patches tab.
  4. Next to AMQ Broker 7.7.0 Operator Installation and Example Files, click Download.

    Download of the amq-broker-operator-7.7.0-ocp-install-examples.zip compressed archive automatically begins.

  5. When the download has completed, move the archive to your chosen installation directory. The following example moves the archive to a directory called ~/broker/operator.

    mkdir ~/broker/operator
    mv amq-broker-operator-7.7.0-ocp-install-examples.zip ~/broker/operator
  6. In your chosen installation directory, extract the contents of the archive. For example:

    cd ~/broker/operator
    unzip amq-broker-operator-7.7.0-ocp-install-examples.zip
  7. Switch to the directory for the Operator archive that you extracted. For example:

    cd amq-broker-operator-7.7.0-ocp-install-examples
  8. Log in to OpenShift Container Platform as a cluster administrator. For example:

    $ oc login -u system:admin
  9. Install the Operator in OperatorHub.

    $ oc create -f deploy/catalog_resources/activemq-artemis-operatorsource.yaml

    After a few minutes, the Operator for AMQ Broker 7.7 is available in the OperatorHub section of the OpenShift Container Platform web console. The name of the Operator is Red Hat Integration - AMQ Broker.

3.2.3. Deploying the Operator from OperatorHub

This procedure shows how to use OperatorHub to deploy the latest version of the Operator for AMQ Broker to a specified OpenShift project.

Important

Deploying the Operator for AMQ Broker using the OperatorHub requires cluster administrator privileges.

Prerequisites

Procedure

  1. Log in to the OpenShift Container Platform web console as a cluster administrator.
  2. In left navigation menu, click OperatorsOperatorHub.
  3. On the Project drop-down menu at the top of the OperatorHub* page, select the project in which you want to deploy the Operator.
  4. On the OperatorHub page, use the Filter by keyword…​ box to find the Red Hat Integration - AMQ Broker Operator.

    Note

    In OperatorHub, you might find more than one Operator than includes AMQ Broker in its name. Ensure that you click the Red Hat Integration - AMQ Broker Operator. When you click this Operator, review the information pane that opens. For AMQ Broker 7.7, the latest version tag of this Operator is 0.17.

  5. Click the Red Hat Integration - AMQ Broker Operator. On the dialog box that appears, click Install.
  6. On the Install Operator page:

    1. Under Update Channel, ensure that the radio button entitled current is selected. This option specifies the channel used to track and receive updates for the Operator. The current value specifies that the AMQ Broker 7.7 channel is used.
    2. Under Installation Mode, ensure that the radio button entitled A specific namespace on the cluster is selected.
    3. From the Installed Namespace drop-down menu, select the project in which you want to install the Operator.
    4. Under Approval Strategy, ensure that the radio button entitled Automatic is selected. This option specifies that updates to the Operator do not require manual approval for installation to take place.
    5. Click Install.

When the Operator installation is complete, the Installed Operators page opens. You should see that the Red Hat Integration - AMQ Broker Operator is installed in the project namespace that you specified.

Additional resources

3.3. Creating Operator-based broker deployments

3.3.1. Deploying a basic broker instance

The following procedure shows how to use a Custom Resource (CR) instance to create a basic broker deployment.

Note

Prerequisites

Procedure

When you have successfully installed the Operator, the Operator is running and listening for changes related to your CRs. This example procedure shows how to use a CR instance to deploy a basic broker in your project.

  1. Start configuring a Custom Resource (CR) instance for the broker deployment.

    1. Using the OpenShift command-line interface:

      1. Log in to OpenShift with administrator privileges for the project in which you are creating the deployment:

        oc login -u <user> -p <password> --server=<host:port>
      2. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted.
    2. Using the OpenShift Container Platform web console:

      1. Log in to the console with administrator privileges for the project in which you are creating the deployment.
      2. Start a new CR instance based on the main broker CRD. In the left pane, click AdministrationCustom Resource Definitions
      3. Click the ActiveMQArtemis CRD.
      4. Click the Instances tab.
      5. Click Create ActiveMQArtemis.

        Within the console, a YAML editor opens, enabling you to configure a CR instance.

    For a basic broker deployment, the configuration might resemble that shown below. This configuration is the default content of the broker_activemqartemis_cr.yaml sample CR.

    apiVersion: broker.amq.io/v2alpha3
    kind: ActiveMQArtemis
    metadata:
      name: ex-aao
      application: ex-aao-app
    spec:
        version: 7.7.0
        deploymentPlan:
            size: 2
            image: registry.redhat.io/amq7/amq-broker:7.7
            ...
    Note

    In the metadata section, you need to include the namespace property and specify a value only if you are using the OpenShift Container Platform web console to create your CR instance. The value that you should specify is the name of the OpenShift project for your broker deployment.

  2. The size value specifies the number of brokers to deploy. The default value of 2 specifies a clustered broker deployment of two brokers. However, to deploy a single broker instance, change the value to 1.
  3. The image value specifies the container image to use to launch the broker. Ensure that this value specifies the latest version of the AMQ Broker 7.7 broker container image in the Red Hat Container Registry, as shown below.

    image: registry.redhat.io/amq7/amq-broker:7.7
    Note

    In the preceding step, the image attribute specifies a floating image tag (that is, 7.7) rather than a full image tag (for example, 7.7-2). When you specify this floating tag, your deployment uses the latest image available in the 7.7 image stream. In addition, when you specify a floating tag such as this, if the imagePullPolicy attribute in your Stateful Set is set to Always, your deployment automatically pulls and uses new micro image versions (for example, 7.7-3, 7-7-4, and so on) when they become available in the Red Hat Container Registry.

  4. Deploy the CR instance.

    1. Using the OpenShift command-line interface:

      1. Save the CR file.
      2. Switch to the project in which you are creating the broker deployment.

        $ oc project <project-name>
      3. Create the CR.

        $ oc create -f <path/to/custom-resource-instance>.yaml
    2. Using the OpenShift Container Platform web console:

      1. When you have finished configuring the CR, click Create.
  5. In the OpenShift Container Platform web console, click WorkloadsStateful Sets (OpenShift Container Platform 4.1 or later) or ApplicationsStateful Sets (OpenShift Container Platform 3.11). You see a new Stateful Set called ex-aao-ss.

    Expand the ex-aao-ss Stateful Set section. You see that there is one Pod, corresponding to the single broker that you defined in the CR.

    On the Events tab of the running Pod, you see that the broker container has started. The Logs tab shows that the broker itself is running.

Additional resources

3.3.2. Deploying clustered brokers

If there are two or more broker Pods running in your project, the Pods automatically form a broker cluster. A clustered configuration enables brokers to connect to each other and redistribute messages as needed, for load balancing.

The following procedure shows you how to deploy clustered brokers. By default, the brokers in this deployment use on demand load balancing, meaning that brokers will forward messages only to other brokers that have matching consumers.

Prerequisites

Procedure

  1. Open the CR file that you used for your basic broker deployment.
  2. For a clustered deployment, ensure that the value of deploymentPlan.size is 2 or greater. For example:

    apiVersion: broker.amq.io/v2alpha3
    kind: ActiveMQArtemis
    metadata:
      name: ex-aao
      application: ex-aao-app
    spec:
        version: 7.7.0
        deploymentPlan:
            size: 4
            image: registry.redhat.io/amq7/amq-broker:7.7
            ...
    Note

    In the metadata section, you need to include the namespace property and specify a value only if you are using the OpenShift Container Platform web console to create your CR instance. The value that you should specify is the name of the OpenShift project for your broker deployment.

  3. Save the modified CR file.
  4. Log in to OpenShift with administrator privileges for the project in which you previously created your basic broker deployment:

    oc login -u <user> -p <password> --server=<host:port>
  5. Switch to the project in which you previously created your basic broker deployment.

    $ oc project <project-name>
  6. At the command line, apply the change:

    $ oc apply -f <path/to/custom-resource-instance>.yaml

    In the OpenShift Container Platform web console, additional broker Pods starts in your project, according to the number specified in your CR. By default, the brokers running in the project are clustered.

  7. Open the Logs tab of each Pod. The logs show that OpenShift has established a cluster connection bridge on each broker. Specifically, the log output includes a line like the following:

    targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@6f13fb88

3.3.3. Applying Custom Resource changes to running broker deployments

The following are some important things to note about applying Custom Resource (CR) changes to running broker deployments:

  • You cannot dynamically update the persistenceEnabled attribute in your CR. To change this attribute, scale your cluster down to zero brokers. Delete the existing CR. Then, recreate and redeploy the CR with your changes, also specifying a deployment size.
  • The value of the deploymentPlan.size attribute in your CR overrides any change you make to size of your broker deployment via the oc scale command. For example, suppose you use oc scale to change the size of a deployment from three brokers to two, but the value of deploymentPlan.size in your CR is still 3. In this case, OpenShift initially scales the deployment down to two brokers. However, when the scaledown operation is complete, the Operator restores the deployment to three brokers, as specified in the CR.
  • As described in Section 3.1.2, “Deploying the Operator using the CLI”, if you create a broker deployment with persistent storage (that is, by setting persistenceEnabled=true in your CR), you might need to provision Persistent Volumes (PVs) for the AMQ Broker Operator to claim for your broker Pods. If you scale down the size of your broker deployment, the Operator releases any PVs that it previously claimed for the broker Pods that are now shut down. However, if you remove your broker deployment by deleting your CR, AMQ Broker Operator does not release Persistent Volume Claims (PVCs) for any broker Pods that are still in the deployment when you remove it. In addition, these unreleased PVs are unavailable to any new deployment. In this case, you need to manually release the volumes. For more information, see Releasing volumes in the OpenShift documentation.
  • In AMQ Broker 7.7, if you want to configure any of the following items, you must add the appropriate configuration to the main CR instance before deploying the CR for the first time.

  • During an active scaling event, any further changes that you apply are queued by the Operator and executed only when scaling is complete. For example, suppose that you scale the size of your deployment down from four brokers to one. Then, while scaledown is taking place, you also change the values of the broker administrator user name and password. In this case, the Operator queues the user name and password changes until the deployment is running with one active broker.
  • All CR changes – apart from changing the size of your deployment, or changing the value of the expose attribute for acceptors, connectors, or the console – cause existing brokers to be restarted. If you have multiple brokers in your deployment, only one broker restarts at a time.