Chapter 8. Known issues

This section describes known issues in AMQ Broker 7.10.

  • ENTMQBR-7359 - Change to current handling of credential secret with 7.10.0 Operator

    The Operator stores the administrator username and password for connecting to the broker in a secret. The default secret name is in the form <custom-resource-name>-credentials-secret. You can create a secret manually or allow the Operator to create a secret.

    If the adminUser and adminPassword attributes are configured in a Custom Resource prior to 7.10.0, the Operator updates a manually-created secret with the values of these attributes. Starting in 7.10.0, the Operator no longer updates a secret that was created manually. Therefore, if you change the values of the adminUser and adminPassword attributes in the CR, you must either:

    • Update the secret with the new username and password
    • Delete the secret and allow the Operator to create a secret. When the Operator creates a secret, it adds the values of the adminUser and adminPassword attributes if these are specified in the CR. If these attributes are not in the CR, the Operator generates random credentials for the secret.

  • ENTMQBR-7363 - redeliveryDelayMultiplier in AddressSettingsType from 7.9 CR cannot be reconciled

    If the redeliveryDelayMultiplier and the redeliveryCollisionAvoidanceFactor attributes are configured in the main broker CR in a 7.8.x or 7.9.x deployment, the new Operator is unable to reconcile any CR after you upgrade to 7.10.x. The reconcile fails because the data type of both attributes changed from float to string in 7.10.x.

    You can work around this issue by deleting the redeliveryDelayMultiplier and the redeliveryCollisionAvoidanceFactor attributes from the spec.deploymentPlan.addressSettings.addressSetting element. Then, configure the attributes in the brokerProperties element. For example: 

    spec:
    ...
    brokerProperties:
    - "addressSettings.#.redeliveryMultiplier=2.1"
    - "addressSettings.#.redeliveryCollisionAvoidanceFactor=1.2"
    Note

    In the brokerProperties element, use the redeliveryMultiplier attribute name instead of the redeliveryDelayMultiplier attribute name that you deleted.

  • ENTMQBR-7396 - [Operator, upgrade] the upgrade to 7.10.1 fails to create new Acceptor/Connector *v1.ServiceAdmission

    After you upgrade from AMQ Broker 7.10.0 to 7.10.1, messaging stops working if incorrect labels added to services and Pod selectors in 7.10.0 were not removed during the upgrade. If messaging is not working after the upgrade, complete the following steps to resolve this issue.

    1. Log in to the OpenShift Container Platform web console as a cluster administrator.
    2. From the Project drop-down menu at the top of the page, select the project in which the Operator is installed.
    3. In the left navigation menu, click Networking → Services.
    4. In the Labels and Pod Selectors columns, check if any of the services has labels other than ActiveMQArtemis and application configured.

    5. For each service that has labels other than ActiveMQArtemis and application configured, complete the following steps to remove the labels.

      1. Click the service to open the Details tab.
      2. In the Labels field, click Edit and delete all labels apart from the ActiveMQArtemis and application labels.
      3. Click Save.
      4. Click the YAML tab.
      5. In the selector element, delete all labels except the ActiveMQArtemis, application and statefulset.kubernetes.io/podname labels.
      6. Click Save.

  • ENTMQBR-7111 - 7.10 versions of operator tend to remove StatefulSet during upgrade

    If you are upgrading to or from AMQ Broker Operator 7.10.0, the new Operator automatically deletes the existing StatefulSet for each deployment during the reconciliation process. When the Operator deletes the StatefulSet, the existing broker pods are deleted, which causes a temporary broker outage.

    You can work around this issue by running the following command to manually delete the StatefulSet and orphan the running pods before the Operator gets to delete the StatefulSet: oc delete statefulset <statefulset-name> --cascade=orphan

    Manually deleting the StatefulSet during the upgrade process allows the new Operator to reconcile the StatefulSet without deleting the running pods. For more information, see Upgrading the Operator using OperatorHub in Deploying AMQ Broker on OpenShift.

  • ENTMQBR-6991 - 7.10-opr-3 does not fix the PV ownerRef for 7.10-opr-2 users

    If you deploy or upgrade to 7.10.0-opr-2 and scale up your deployment, the new PVs are created with an ownerReference attribute that can cause data loss if you later delete the deployment CR. For example, if you deploy 7.10.0-opr-1, upgrade to 7.10.0-opr-2 and then scale from 3 to 4 broker instances, you could lose data if you delete the ActiveMQArtemis CR.

    To workaround this issue, you can:

    • Skip the 7.10.0-opr-2 upgrade if possible.
    • Avoid scaling up your deployment while the 7.10.0-opr-2 release is active in your cluster. You can scale up after you deploy 7.10.0-opr-3.
    • Avoid deleting the deployment CR until a later release resolves this issue.
    • Manually remove the ownerReference value for the PVs affected.

  • ENTMQBR-6712 - Taints and Tolerations - "tolerationSeconds" breaks the deployment

    If you add a tolerationSeconds attribute in the tolerations section of the CR, the Operator reconciliation process does not work, and broker pods are not scheduled correctly. To work around this issue, do not add a tolerationSeconds attribute in the tolerations section of the CR.

  • ENTMQBR-6473 - Incompatible configuration due to schema URL change

    When you try to use a broker instance configuration from a previous release with a version 7.9 or 7.10 instance, an incompatible configuration as a result of a schema URL change causes the broker to crash. To work around this issue, update the schema URL in the relevant configuration files as outlined in Upgrading from 7.9.0 to 7.10.0 on Linux.

  • ENTMQBR-4813 AsynchronousCloseException with large messages and multiple C++ subscribers

    If multiple C++ Publisher clients that uses the AMQP protocol are running on the same host as subscribers and the broker, and a publisher sends a large message, one of the subscribers crashes.

  • ENTMQBR-6655 - The command artemis check queue fails with "Could not start Jolokia agent"

    Before it runs, the artemis check queue command displays the following error message: Could not start Jolokia agent: java.lang.IllegalStateException: Cannot open keystore for https communication: java.net.BindException: Address already in use.

  • ENTMQBR-6654 - requireLogin:true works only for a new broker CR applied and not for existing one.

    If the requireLogin property is set to true in the CR, the AMQ_REQUIRE_LOGIN environment variable is not updated in the stateful set of existing broker instances and the console credential are not validated. To work around this issue, manually update the environment variable value in the stateful set for the existing instance.

  • ENTMQBR-5936 - The client doesn’t failover to the backup server if the URL targets non clustered ports.

    If the connection URL that a client uses to connect to a HA cluster has a port that is not configured in the broker’s static-connectors, after a failover occurs, the client retries the connection to the previously live broker and does not attempt to connect to the new live broker.

  • ENTMQBR-6728 - Upgrade path is broken

    This issue prevents AMQ Broker 7.9 users who are subscribed to the 7.x channel from automatically upgrading to AMQ Broker 7.10. To workaround this issue, subscribe to the 7.10.x channel.

  • ENTMQBR-5749 - Remove unsupported operators that are visible in OperatorHub

    Only the Operators and Operator channels mentioned in Deploying the Operator from OperatorHub are supported. For technical reasons associated with Operator publication, other Operator and channels are visible in the OperatorHub and should be ignored. For reference, the following list shows which Operators are visible, but not supported:

    • Red Hat Integration - AMQ Broker LTS - all channels
    • Red Hat Integration - AMQ Broker - alpha, current, and current-76

  • ENTMQBR-17 - AMQ222117: Unable to start cluster connection

    A broker cluster may fail to initialize properly in environments that support IPv6. The failure is due to a SocketException that is indicated by the log message Can’t assign requested address. To work around this issue, set the java.net.preferIPv4Stack system property to true.

  • ENTMQBR-520 - Receiving from address named the same as a queue bound to another address should not be allowed

    A queue with the same name as an address must only be assigned to address. Creating a queue with the same name as an existing address, but bound to an address with a different name, is an invalid configuration. Doing so can result in incorrect messages being routed to the queue.

  • ENTMQBR-569 - Conversion of IDs from OpenWire to AMQP results in sending IDs as binary

    When communicating cross-protocol from an A-MQ 6 OpenWire client to an AMQP client, additional information is encoded in the application message properties. This is benign information used internally by the broker and can be ignored.

  • ENTMQBR-636 - Journal breaks, causing JavaNullPointerException, under perf load (mpt)

    To prevent IO-related issues from occurring when the broker is managing heavy loads, verify that the JVM is allocated with enough memory and heap space. See the section titled "Tuning the VM" in the Performance Tuning chapter of the ActiveMQ Artemis documentation.

  • ENTMQBR-648 - JMS Openwire client is unable to send messages to queue with defined purgeOnNoConsumer or queue filter

    Using an A-MQ 6 JMS client to send messages to an address that has a queue with purgeOnNoConsumer set to true fails if the queue has no consumers. It is recommended that you do not set the purgeOnNoConsumer option when using A-MQ 6 JMS clients.

  • ENTMQBR-652 - List of known amq-jon-plugin bugs

    This version of amq-jon-plugin has known issues with the MBeans for broker and queue.

    Issues with the broker MBean:

    • Closing a connection throws java.net.SocketTimeoutException exception
    • listSessions() throws java.lang.ClassCastException
    • Adding address settings throws java.lang.IllegalArgumentException
    • getConnectorServices() operation cannot be found
    • listConsumersAsJSON() operation cannot be found
    • getDivertNames() operation cannot be found
    • Listing network topology throws IllegalArgumentException
    • Remove address settings has wrong parameter name

    Issues with the queue MBean:

    • expireMessage() throws argument type mismatch exception
    • listDeliveringMessages() throws IllegalArgumentException
    • listMessages() throws java.lang.Exception
    • moveMessages() throws IllegalArgumentException with error message argument type mismatch
    • removeMessage() throws IllegalArgumentException with error message argument type mismatch
    • removeMessages() throws exception with error Can’t find operation removeMessage with 2 arguments
    • retryMessage() throws argument type mismatch IllegalArgumentException

  • ENTMQBR-655 - [AMQP] Unable to send message when populate-validated-user is enabled

    The configuration option populate-validated-user is not supported for messages produced using the AMQP protocol.

  • ENTMQBR-897 - Openwire client/protocol issues with special characters in destination name

    Currently AMQ OpenWire JMS clients cannot access queues and addresses that include the following characters in their name: comma (','), hash ('#'), greater than ('>'), and whitespace.

  • ENTMQBR-944 - [A-MQ7, Hawtio, RBAC] User gets no feedback if operation access was denied by RBAC

    The console can indicate that an operation attempted by an unauthorized user was successful when it was not.

  • ENTMQBR-1875 - [AMQ 7, ha, replicated store] backup broker appear not to go "live" or shutdown after - ActiveMQIllegalStateException errorType=ILLEGAL_STATE message=AMQ119026: Backup Server was not yet in sync with live

    Removing the paging disk of a master broker while a backup broker is trying to sync with the master broker causes the master to fail. In addition, the backup broker cannot become live because it continues trying to sync with the master.

  • ENTMQBR-2068 - some messages received but not delivered during HA fail-over, fail-back scenario

    Currently, if a broker fails over to its slave while an OpenWire client is sending messages, messages being delivered to the broker when failover occurs could be lost. To work around this issue, ensure that the broker persists the messages before acknowledging them.

  • ENTMQBR-3331 - Stateful set controller can’t recover from CreateContainerError, blocking the operator

    If the AMQ Broker Operator creates a stateful set from a Custom Resource (CR) that has a configuration error, the stateful set controller is unable to roll out the updated stateful set when the error is resolved.

    For example, a misspelling in the value of the image attribute in your main broker CR causes the status of the first Pod created by the stateful set controller to remain Pending. If you then fix the misspelling and apply the CR changes, the AMQ Broker Operator updates the stateful set. However, a Kubernetes known issue prevents the stateful set controller from rolling out the updated stateful set. The controller waits indefinitely for the Pod that has a Pending status to become Ready, so the new Pods are not deployed.

    To work around this issue, you must delete the Pod that has a Pending status to allow the stateful set controller to deploy the new Pods. To check which Pod has a Pending status, use the following command: oc get pods --field-selector=status.phase=Pending. To delete a Pod, use the oc delete pod <pod name> command.

  • ENTMQBR-3846 - MQTT client does not reconnect on broker restart

    When you restart a broker, or a broker fails over, the active broker does not restore connections for previously-connected MQTT clients. To work around this issue, to reconnect an MQTT client, you need to manually call the subscribe() method on the client.

  • ENTMQBR-4023 - AMQ Broker Operator: Pod Status pod names do not reflect the reality

    For an Operator-based broker deployment in a given OpenShift project, if you use the oc get pod command to list the broker Pods, the ordinal values for the Pods start at 0, for example, amq-operator-test-broker-ss-0. However, if you use the oc describe command to get the status of broker Pods created from the activemqartmises Custom Resource (that is, oc describe activemqartemises), the Pod ordinal values incorrectly start at 1, for example, amq-operator-test-broker-ss-1. There is no way to work around this issue.

  • ENTMQBR-4127 - AMQ Broker Operator: Route name generated by Operator might be too long for OpenShift

    For each broker Pod in an Operator-based deployment, the default name of the Route that the Operator creates for access to the AMQ Broker management console includes the name of the Custom Resource (CR) instance, the name of the OpenShift project, and the name of the OpenShift cluster. For example, my-broker-deployment-wconsj-0-svc-rte-my-openshift-project.my-openshift-domain. If some of these names are long, the default Route name might exceed the limit of 63 characters that OpenShift enforces. In this case, in the OpenShift Container Platform web console, the Route shows a status of Rejected.

    To work around this issue, use the OpenShift Container Platform web console to manually edit the name of the Route. In the console, click the Route. On the Actions drop-down menu in the top-right corner, select Edit Route. In the YAML editor, find the spec.host property and edit the value.

  • ENTMQBR-4140 - AMQ Broker Operator: Installation becomes unusable if storage.size is improperly specified

    If you configure the storage.size property of a Custom Resource (CR) instance to specify the size of the Persistent Volume Claim (PVC) required by brokers in a deployment for persistent storage, the Operator installation becomes unusable if you do not specify this value properly. For example, suppose that you set the value of storage.size to 1 (that is, without specifying a unit). In this case, the Operator cannot use the CR to create a broker deployment. In addition, even if you remove the CR and deploy a new version with storage.size specified correctly, the Operator still cannot use this CR to create a deployment as expected.

    To work around this issue, first stop the Operator. In the OpenShift Container Platform web console, click Deployments. For the Pod that corresponds to the AMQ Broker Operator, click the More options menu (three vertical dots). Click Edit Pod Count and set the value to 0. When the Operator Pod has stopped, create a new version of the CR with storage.size correctly specified. Then, to restart the Operator, click Edit Pod Count again and set the value back to 1.

  • ENTMQBR-4141 - AMQ Broker Operator: Increasing Persistent Volume size requires manual involvement even after recreating Stateful Set

    If you try to increase the size of the Persistent Volume Claim (PVC) required by brokers in a deployment for persistent storage, the change does not take effect without further manual steps. For example, suppose that you configure the storage.size property of a Custom Resource (CR) instance to specify an initial size for the PVC. If you modify the CR to specify a different value of storage.size, the existing brokers continue to use the original PVC size. This is the case even if you scale the deployment down to zero brokers and then back up to the original number. However, if you scale the size of the deployment up to add additional brokers, the new brokers use the new PVC size.

    To work around this issue, and ensure that all brokers in the deployment use the same PVC size, use the OpenShift Container Platform web console to expand the PVC size used by the deployment. In the console, click Storage → Persistent Volume Claims. Click your deployment. On the Actions drop-down menu in the top-right corner, select Expand PVC and enter a new value.