Chapter 1. Upgrading Fuse Online

The Fuse Online upgrade process depends on whether Fuse Online is installed on Red Hat OpenShift Online or on OpenShift Container Platform (OCP).

1.1. Access Docker images before an upgrade

By default, the Fuse Online upgrade process pulls Docker images for the Fuse Online backup and restore database from the docker.io registry, instead of pulling them from registry.redhat.io (as described in the known issue https://issues.redhat.com/browse/ENTESB-15364). Because docker.io imposes a service limitation on free downloads, the Fuse Online upgrade process could encounter the following error when it attempts to download the backup and restore database images: 

error: code = Unknown desc = toomanyrequests: You have reached your pull rate limit

For more information about the Docker Hub pull rate limit, see the Docker documentation (https://www.docker.com/increase-rate-limits).

To avoid encountering the Docker limit error, before you start the Fuse Online upgrade process, make sure that you have access to Docker images. How you access the Docker images depends on your OpenShift Container Platform (OCP) version:

  • For OCP 4.6, add Docker Hub credentials into your existing syndesis pull secret as described in Accessing Docker images on OCP 4.6. When you add your Docker credentials to the syndesis pull secret, the Docker Hub limit on API is raised, depending on your Docker Hub account type.
  • For OCP 3.11, pull the Fuse Online backup and restore database images from docker.io to a local cache as described in Accessing Docker images on OCP 3.11.

1.1.1. Accessing Docker images on OCP4

To avoid encountering a possible Docker limit error, before you start the Fuse Online upgrade process, add your docker credentials to the syndesis pull secret.

Prerequisite

You have OpenShift cluster administrator access.

Procedure

To add Docker Hub credentials into your existing syndesis pull secret:

  1. Log into OpenShift with your administrator credentials: 

    oc login -u system:admin

  2. Retrieve your syndesis pull secret by running the following command: 

     oc get secrets syndesis-pull-secret -o=custom-columns=SECRET:.data.* --no-headers | base64 -d | jq

    This command returns output similar to the following, where <AUTH> is your encoded username and password in base64: 

     {
   "auths":{
      "registry.redhat.io":{
         "username":"<AUTH>",
         "password":"<AUTH>",
         "auth":"<AUTH>"
      },
      <You can have more auths elements here>
   }
  }

  3. To add your Docker Hub registry credentials, edit your pull secret and save it to the PULL_SECRET variable: 

     PULL_SECRET=$(base64 -w 0 <<EOF
{
   "auths":{
      "registry.redhat.io":{
         "username":"<AUTH>",
         "password":"<AUTH>",
         "auth":"<AUTH>"
      },
      <You can have more auths elements here>
      },
      "https://index.docker.io/v1/": {
         "auth": "<AUTH>"
      }
   }
}
EOF
)

  4. To patch your syndesis pull secret, run the following command: 

    oc apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: syndesis-pull-secret
data:
  .dockerconfigjson: $PULL_SECRET
type: kubernetes.io/dockerconfigjson
EOF

Next step

1.1.2. Accessing Docker images on OCP 3.11

To avoid encountering a possible Docker limit error, before you start the Fuse Online upgrade process on OCP 3.11, you should pull the required images from docker.io to a local cache.

Prerequisite

You have OpenShift cluster administrator access.

Procedure

  1. Login to the OpenShift node with your OpenShift username and the OpenShift cluster URL: 

    ssh login <user>@node1.<clusterUrl>

  2. Login to docker with your username and password: 

    sudo docker login -u <username> -p <password> docker.io

  3. Run the following commands that pull the PostgreSQL images: 

    sudo docker pull crunchydata/crunchy-pgdump:centos7-10.11-4.2.1
sudo docker pull crunchydata/crunchy-pgrestore:centos7-10.11-4.2.1
sudo docker pull centos:7

    The PostgresQL images are pulled to the OpenShift node’s internal docker registry.

  4. Repeat the previous three steps for each OpenShift node.

Next step

Upgrade Fuse Online by following the steps in Upgrading Fuse Online by using the install script.

1.2. Upgrading Fuse Online by using the OperatorHub

Fuse Online 7.8 is available in the OperatorHub starting with OCP 4.6. If you are using OCP 4.5, you must upgrade to OCP 4.6 if you want to install Fuse Online 7.8.

Note: If you used the install script to install Fuse Online 7.7, you should use the install script to upgrade to Fuse Online 7.8 as described in Upgrading Fuse Online by using the install script.

When you install Fuse Online, you specify a channel with the fuse-online-v7.n format, where n is the current release number. For example, for Fuse Online 7.8, the channel is fuse-online-v7.8.

Upgrading from a Fuse Online 7.8 version to a newer Fuse Online 7.8 version depends on the Approval Strategy that you selected when you installed Fuse Online:

  • For Automatic updates, when a new version of the Fuse Online operator is available, the OpenShift Operator Lifecycle Manager (OLM) automatically upgrades the running instance of the Fuse Online without human intervention.
  • For Manual updates, when a newer version of an Operator is available, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to have the Fuse Online operator updated to the new version as described in the Manually approving a pending Operator upgrade section of the OpenShift documentation.

During and after an infrastructure upgrade, existing integrations continue to run with the older versions of Fuse Online libraries and dependencies. To have them run with the updated versions, you must republish them.

For OCP 4.6 and later, to upgrade from Fuse Online 7.7 to 7.8, use the steps in the following procedure.

Procedure

  1. To avoid encountering Docker limit errors, before you start the Fuse Online upgrade process, add your docker credentials to the syndesis pull secret as described in as described in Access Docker images before an upgrade.

  2. To upgrade the Fuse Online operator:

    1. In the OpenShift web console, click Operators > Installed Operators.
    2. Click the Fuse Online operator and then Subscription.
    3. Next to the Channel, click the Edit icon.
    4. Select the fuse-online-v7.8 channel and then click Save.

If you specified Manual updates when you installed Fuse Online, approve the Operator update request by following the instructions in the Manually approving a pending Operator upgrade section of the OpenShift documentation.

1.3. Upgrading Fuse Online by using the install script

If you installed Fuse Online by using the install script (rather than the OperatorHub), here are the general steps to upgrade Fuse Online:

  • A cluster administrator sets up access to Docker images for the the Fuse Online backup and restore database:

    • For OCP 3.11, pull the Fuse Online backup and restore database images from docker.io to a local cache.
    • For OCP 4.6, add Docker Hub credentials into the existing syndesis pull secret.
  • Download the latest Fuse Online release.
  • Obtain permission to upgrade Fuse Online from a cluster administrator.
  • Run the update script.

The upgrade procedure for the following upgrades is the same:

  • From Fuse Online 7.7 to Fuse Online 7.8
  • From a Fuse Online 7.8 version to a newer Fuse Online 7.8 version

Prerequisites

  • You installed and are running version 7.7 of Fuse Online on OCP on-site. OR, you installed and are running a version of 7.8 of Fuse Online on OCP 3.11 and you want to upgrade to fresh application images.

    For earlier versions:

    • If you are running version 7.6 of Fuse Online on OCP, then you must upgrade to 7.7 and then you can upgrade to 7.8.
    • If you are running version 7.5 of Fuse Online on OCP, then you must upgrade to 7.6 and then you can upgrade to 7.7.
    • If you are running version 7.4 of Fuse Online on OCP, then you must upgrade to 7.5 and then you can upgrade to 7.6.
    • If you are running version 7.3 of Fuse Online on OCP, then you must upgrade to 7.4 and then you can upgrade to 7.5.
    • If you are running version 7.2 of Fuse Online on OCP, then you must upgrade to 7.3.
    • If you are running version 7.1 of Fuse Online on OCP, then you must upgrade to 7.2.
  • You installed the oc client tool and it is connected to the OCP cluster in which Fuse Online is installed.
  • You have cluster administration permissions, which are required for the first two steps in this procedure.

Procedure

  1. To avoid encountering a possible Docker limit error, a cluster administrator sets up access to Docker images as described in Access Docker images before an upgrade.

  2. A cluster administrator downloads the Fuse Online package and grants permission for a user to upgrade Fuse Online in a particular project:

    1. Download the package containing the Fuse Online installation scripts from the following location:

      https://github.com/syndesisio/fuse-online-install/releases/tag/1.11

      Unpack the downloaded archive at a convenient location on your file system. The fuse-online-install-1.11 directory contains the scripts and supporting files for upgrading Fuse Online.

    2. Change to the directory that contains the extracted archive. For example:

      cd fuse-online-install-1.11

    3. Log in to OpenShift with a cluster administration account, for example:

      oc login -u admin -p admin

    4. Change to the OpenShift project in which Fuse Online needs to be upgraded, for example:

      oc project fuse-online-project

    5. Update the Fuse Online custom resource definition:

      bash install_ocp.sh --setup

    6. Grant permission for upgrading Fuse Online in just this project. For example, the following command grants permission for upgrading Fuse Online to the developer user. After the cluster administrator runs this command, the developer user can upgrade Fuse Online in only this project, which is fuse-online-project, in this example:

      bash install_ocp.sh --grant developer

  3. The user who was granted permission to upgrade Fuse Online performs the upgrade:

    1. Log in to OpenShift, for example:

      oc login -u developer

    2. Switch to the project in which you want to upgrade Fuse Online, for example:

      oc project fuse-online-project

    3. To check which version you are about to upgrade to, run the update script with the --version option, as follows:

      bash update_ocp.sh --version

    4. Invoke the update script as follows:

      bash update_ocp.sh

      To learn more about the script, invoke bash update_ocp.sh --help.

      During and after an infrastructure upgrade, existing integrations continue to run with the older versions of Fuse Online libraries and dependencies.

  4. Upgrade Fuse Online integrations that are running as follows:

    1. In Fuse Online, select the integration that you want to upgrade.
    2. Select Edit.
    3. Select Publish to republish the integration.

    Republishing the integration forces a rebuild that uses the latest Fuse Online dependencies.

1.4. Camel migration considerations

Red Hat Fuse uses Apache Camel 2.23. You should consider the following updates to Camel 2.22 and 2.23 when you upgrade to Fuse 7.8.

Camel 2.22 updates

  • Camel has upgraded from Spring Boot v1 to v2 and therefore v1 is no longer supported.
  • Upgraded to Spring Framework 5. Camel should work with Spring 4.3.x as well, but going forward Spring 5.x will be the minimum Spring version in future releases.
  • Upgraded to Karaf 4.2. You may run Camel on Karaf 4.1 but we only officially support Karaf 4.2 in this release.
  • Optimized using toD DSL to reuse endpoints and producers for components where it is possible. For example, HTTP based components will now reuse producer (HTTP clients) with dynamic URIs sending to the same host.
  • The File2 consumer with read-lock idempotent/idempotent-changed can now be configured to delay the release tasks to expand the window when a file is regarded as in-process, which is usable in active/active cluster settings with a shared idempotent repository to ensure other nodes don’t too quickly see a processed file as a file they can process (only needed if you have readLockRemoveOnCommit=true).
  • Allow to plugin a custom request/reply correlation id manager implementation on Netty4 producer in request/reply mode. The Twitter component now uses extended mode by default to support tweets greater than 140 characters
  • Rest DSL producer now supports being configured in REST configuration by using endpointProperties.
  • The Kafka component now supports HeaderFilterStrategy to plugin custom implementations for controlling header mappings between Camel and Kafka messages.
  • REST DSL now supports client request validation to validate that Content-Type/Accept headers are possible for the REST service.
  • Camel now has a Service Registry SPI which allows you to register routes to a service registry (such as consul, etcd, or zookeeper) by using a Camel implementation or Spring Cloud.
  • The SEDA component now has a default queue size of 1000 instead of unlimited.

  • The following noteworthy issues have been fixed:

    • Fixed a CXF continuation timeout issue with camel-cxf consumer that could cause the consumer to return a response with data instead of triggering a timeout to the calling SOAP client.
    • Fixed camel-cxf consumer doesn’t release UoW when using a robust one-way operation.
    • Fixed using AdviceWith and using weave methods on onException etc. not working.
    • Fixed Splitter in parallel processing and streaming mode may block, while iterating message body when the iterator throws an exception in the first invoked next() method call.
    • Fixed Kafka consumer to not auto commit if autoCommitEnable=false.
    • Fixed file consumer was using markerFile as read-lock by default, which should have been none.
    • Fixed using manual commit with Kafka to provide the current record offset and not the previous (and -1 for first).
    • Fixed Content Based Router in Java DSL may not resolve property placeholders in when predicates.

Camel 2.23 updates

  • Upgraded to Spring Boot 2.1.
  • Additional component-level options can now be configured by using spring-boot auto-configuration. These options are included in the spring-boot component metadata JSON file descriptor for tooling assistance.
  • Added a documentation section that includes all the Spring Boot auto configuration options for all the components, data-formats, and languages.
  • All the Camel Spring Boot starter JARs now include META-INF/spring-autoconfigure-metadata.properties file in their JARs to optimize Spring Boot auto-configuration.
  • The Throttler now supports correlation groups based on dynamic expression so that you can group messages into different throttled sets.
  • The Hystrix EIP now allows inheritance for Camel’s error handler so that you can retry the entire Hystrix EIP block again if you have enabled error handling with redeliveries.
  • SQL and ElSql consumers now support dynamic query parameters in route form. Note that this feature is limited to calling beans by using simple expressions.
  • The swagger-restdsl maven plugin now supports generating DTO model classes from the Swagger specification file.

  • The following noteworthy issues have been fixed:

    • The Aggregator2 has been fixed to not propagate control headers for forcing completion of all groups, so it will not happen again if another aggregator EIP is in use later during routing.
    • Fixed Tracer not working if redelivery was activa†ed in the error handler.
    • The built-in type converter for XML Documents may output parsing errors to stdout, which has now been fixed to output by using the logging API.
    • Fixed SFTP writing files by using the charset option would not work if the message body was streaming-based.
    • Fixed Zipkin root id to not be reused when routing over multiple routes to group them together into a single parent span.
    • Fixed optimized toD when using HTTP endpoints had a bug when the hostname contains an IP address with digits.
    • Fixed issue with RabbitMQ with request/reply over temporary queues and using manual acknowledge mode. It would not acknowledge the temporary queue (which is needed to make request/reply possible).
    • Fixed various HTTP consumer components that may not return all allowed HTTP verbs in Allow header for OPTIONS requests (such as when using rest-dsl).
    • Fixed the thread-safety issue with FluentProducerTemplate.