Chapter 2. Fuse Online

Fuse Online provides a web browser interface that lets a business expert integrate two or more different applications or services without writing code. It also provides features that allow the addition of code if it is needed for complex use cases.

Fuse Online runs an integration on OpenShift as a Spring Boot application that uses Apache Camel.

2.1. About Fuse Online distributions

Fuse Online is Red Hat’s web-based integration platform. Syndesis is the open source project for Fuse Online. Fuse Online runs in these OpenShift environments:

Host Environment

Installation

OpenShift Dedicated

Red Hat installs and provisions Fuse Online on Red Hat infrastructure.

OpenShift Container Platform

Customer installs and manages.

2.2. New features in Fuse Online 7.10

Fuse Online 7.10 provides the following new features:

  • Support for connecting to Red Hat Managed Kafka

    To support connecting to Red Hat Managed Kafka, the Kafka Message Broker connector includes the following fields:

    • Security Protocol
    • SASL Mechanism
    • Username
    • Password
    • SASL Login Callback Handler Class
    • OAuth Token Endpoint URI

      For more information, see Chapter 15, "Connecting to Kafka" in Connecting Fuse Online to Applications and Services.

  • Integration labels

    When you save an integration, you can apply one or more labels to it. A label is a key-value pair tag that you can apply to an integration (or other OpenShift resource) for later filtering and selecting in OpenShift. For example, an OpenShift admin user can filter the list of running pods or deployments by label.

  • Integration environment variables

    Optionally, when you save an integration, you can set one or more environment variables that Fuse Online applies to the integration Pod. You can use these environment variables to set Spring Boot configuration options, for example, to set SERVER_MAX_HTTP_HEADER_SIZE. Existing environment variables set manually are preserved.

    Note that setting these environment variables in Fuse Online when you save an integration does not change or impact any other environment settings set manually, for example, through the OpenShift web console interface.

  • Maven mirror setting

    You can specify the Maven repository that you want Fuse Online to use for accessing Maven artifacts when it builds integrations. In the Syndesis custom resource, specify the Maven repository as the value of the components:server:features:maven:mirror setting.

2.3. Changes in Fuse Online 7.10

Fuse Online 7.10 changes Fuse Online 7.9 features as follows:

  • To enable the sample Postgres database, use the todo addon option

    In previous releases, if you want to include a sample database and sample database connector in your Fuse Online installation, set the demoData option in the Fuse Online custom resource. Starting with Fuse Online 7.10, you enable the sample database and connector by setting the todo addon option, which also includes the sample Todo app for testing integrations.

  • Kafka broker URIs are auto-discovered for AMQ Streams

    If you use AMQ Streams (API versions v1beta1 or v1beta2), when you create a new Kafka Message Broker connection in Fuse Online, the Kafka Broker URI is auto-discovered and shown in the Kafka Broker URI drop-down list.

  • Disconnected environment requires setting environment variables

    For Fuse Online to install and work in a disconnected environment, you must set the following environment variables to syndesis-oauthproxy:

    • HTTPS_PROXY
    • HTTP_PROXY
    • NO_PROXY
  • deployIntegrations flag no longer available

    The deployIntegrations option in the Fuse Online custom resource that controlled whether integrations were deployed has been removed.

2.4. Upgrading Fuse Online integrations

To upgrade a Fuse Online environment that is running on OCP on-site, you must update Fuse Online by using the operator and then republish any running integrations as described in Upgrading Fuse Online.

On OCP 4.9, when you upgrade to 7.10 by using the operator, the following warning is displayed during the Fuse Online Operator upgrade process:

W1219 18:38:58.064578 1 warnings.go:70] extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

This warning appears because clients (that Fuse Online uses for the Kubernetes/OpenShift API initialization code) access a deprecated Ingress version. This warning is not an indicator of complete use of deprecated APIs and there is no issue with upgrading to Fuse Online 7.10.

2.5. Important notes for Fuse Online

Important notes for the Fuse 7.10 release of the Fuse Online distribution:

  • Fuse Online no longer supports Camel K runtime or the KNative connector.
  • When Fuse Online is installed and provisioned on Red Hat infrastructure, the account is limited to a specific number of integrations that can be running at one time. For details, see the pricing plan.
  • An OpenAPI schema that you upload to Fuse Online might not define input/output types. When Fuse Online creates a custom API client from an OpenAPI schema that does not specify input/output types then it is not possible to create an integration that maps integration data to fields that the API client can process or from fields that the API client processed. If an integration requires data mapping to or from a custom API, then when you upload the OpenAPI schema, click Review/Edit to open API Designer, which is an API editing tool, and add input/output type specifications.
  • Since Fuse 7.8, an OpenAPI document that you use for a custom API client connector or for an API provider integration cannot have cyclic schema references. For example, a JSON schema that specifies a request or response body cannot reference itself as a whole nor reference any part of itself through any number of intermediate schemas.
  • On OCP 4.9 (or later), the application-monitoring project no longer works. It is a prerequisite for monitoring Fuse Online integrations and infrastructure components with Prometheus and Grafana.

    To workaround this issue, you can use the built-in OpenShift monitoring stack (in the openshift-monitoring namespace) to use the openshift-user-workload-monitoring feature and the grafana-operator to use the ops addon as described in the following Adding Fuse Online monitoring resources (Prometheus and Grafana) on OCP 4.9 (or later) procedure.

2.5.1. Adding Fuse Online monitoring resources (Prometheus and Grafana) on OCP 4.9 (or later)

Prerequisites

  • Fuse Online is installed and running on OCP 4.9 (or later) on-site.
  • The oc client tool is installed and it is connected to the OCP cluster in which Fuse Online is installed.
  • You have admin access to the OCP cluster.
  • Your Fuse Online installation is configured with the ops addon enabled. If required, you can enable it with this command:

    oc patch syndesis/app --type=merge -p '{"spec": {"addons": {"ops": {"enabled": true}}}}'

Procedure

  1. If there is an existing openshift-monitoring configuration, skip to Step 2.

    Otherwise, create an openshift-monitoring configuration, that sets the user workload monitoring option to true and then skip to Step 3:

    oc apply -f - <<EOF
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: cluster-monitoring-config
      namespace: openshift-monitoring
    data:
      config.yaml:
       enableUserWorkload: true
    EOF
  2. If there is an existing openshift-monitoring configuration:

    1. Check the existing openshift-monitoring configuration to determine whether the user workload monitoring option is set to true:

      oc get -n openshift-monitoring cm/cluster-monitoring-config -ojsonpath='{.data.config\.yaml}'

      If the result is enableUserWorkload: true, the user workload monitoring option is set to true. Skip to Step 3.

      If the result shows any other configurations, continue to the next step to enable the monitoring of user workloads by editing the ConfigMap.

    2. Open the ConfigMap file in an editor, for example:

      oc -n openshift-monitoring edit cm/cluster-monitoring-config
    3. Set enableUserWorkload to true. For example:

      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: cluster-monitoring-config
        namespace: openshift-monitoring
      data:
        config.yaml:
         enableUserWorkload: true
    4. Save the ConfigMap file.
  3. Use the following command to watch the status of the pods in the openshift-user-workload-monitoring namespace:

    oc -n openshift-user-workload-monitoring get pods -w

    Wait until the status of the pods is Running, for example:

    prometheus-operator-5d989f48fd-2qbzd   2/2     Running
    prometheus-user-workload-0             5/5     Running   prometheus-user-workload-1             5/5     Running
    thanos-ruler-user-workload-0           3/3     Running
    thanos-ruler-user-workload-1           3/3     Running
  4. Verify that the Fuse Online alert rules are enabled in Prometheus:

    1. Access the internal prometheus instance

      oc port-forward -n openshift-user-workload-monitoring pod/prometheus-user-workload-0 9090
    2. Open your browser to localhost:9090
    3. Select Status> Targets. You should see three syndesis endpoints.
    4. Press CTRL-C to terminate the port-forward process.
  5. From the OperatorHub, install the Grafana Operator 4.1.0 to a namespace of your choosing, for example, to the grafana-middleware namespace.
  6. Add a cluster role and a cluster role binding to allow the grafana-operator to list nodes and namespaces:

    1. Download the cluster role YAML file from the grafana-operator website:

      curl https://raw.githubusercontent.com/grafana-operator/grafana-operator/master/deploy/cluster_roles/cluster_role_grafana_operator.yaml > tmp_role.yaml
    2. Add cluster permission for the grafana-operator to read other namespaces and nodes:

      cat <<EOF >> tmp_role.yaml
        - apiGroups:
            - ""
          resources:
            - namespaces
            - nodes
          verbs:
            - get
            - list
            - watch
      EOF
      oc apply -f tmp_role.yaml
      oc apply -f - <<EOF
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: grafana-operator
      roleRef:
        name: grafana-operator
        kind: ClusterRole
        apiGroup: ""
      subjects:
        - kind: ServiceAccount
          name: grafana-operator-controller-manager
          namespace: grafana-middleware
      EOF
  7. Enable the grafana-operator to read Grafana dashboards from other namespaces by using the DASHBOARD_NAMESPACES_ALL environment variable to limit the namespaces:

    oc -n grafana-middleware patch subs/grafana-operator --type=merge -p '{"spec":{"config":{"env":[{"name":"DASHBOARD_NAMESPACES_ALL","value":"true"}]}}}'
  8. Check that the grafana pods are recreated:

    oc -n grafana-middleware get pods -w
  9. Optionally, view the grafana-operator logs:

    oc -n grafana-middleware logs -f `oc -n grafana-middleware get pods -oname|grep grafana-operator-controller-manager` -c manager
  10. Add a Grafana custom resource to start a Grafana server pod, for example:

    oc apply -f - <<EOF
    apiVersion: integreatly.org/v1alpha1
    kind: Grafana
    metadata:
      name: grafana-middleware
      namespace: grafana-middleware
    spec:
      config:
        auth:
          disable_signout_menu: true
        auth.anonymous:
          enabled: true
        log:
          level: warn
          mode: console
        security:
          admin_password: secret
          admin_user: root
      dashboardLabelSelector:
      - matchExpressions:
        - key: app
          operator: In
          values:
          - grafana
          - syndesis
      ingress:
        enabled: true
    EOF
  11. Allow the grafana-operator to read monitoring information:

    oc -n grafana-middleware adm policy add-cluster-role-to-user cluster-monitoring-view -z grafana-serviceaccount
  12. Add a GrafanaDatasource to query thanos-querier:

    oc apply -f - <<EOF
    apiVersion: integreatly.org/v1alpha1
    kind: GrafanaDataSource
    metadata:
      name: prometheus-grafanadatasource
      namespace: grafana-middleware
    spec:
      datasources:
        - access: proxy
          editable: true
          isDefault: true
          jsonData:
            httpHeaderName1: 'Authorization'
            timeInterval: 5s
            tlsSkipVerify: true
          name: Prometheus
          secureJsonData:
          httpHeaderValue1: "Bearer $(oc -n grafana-middleware serviceaccounts get-token grafana-serviceaccount)"
          type: prometheus
          url: "https://$(oc get route thanos-querier -n openshift-monitoring -ojsonpath='{.spec.host}')"
      name: prometheus-grafanadatasource.yaml
    EOF
  13. View the grafana server log:

    oc logs -f `oc get pods -l app=grafana -oname`
  14. Access the grafana URL and view the Fuse Online dashboards:

    echo "https://"$(oc -n grafana-middleware get route/grafana-route -ojsonpath='{.spec.host}')

2.6. Obtaining technical support for Fuse Online

To obtain technical support, in the Fuse Online console, in the left navigation panel, click Support. Use the Support page to download diagnostic information for all integrations or for one or more integrations that you choose. The page also provides a link for opening a support ticket and providing the diagnostic information that you downloaded.

2.7. Technology Preview features in Fuse Online

This release includes the Technology Preview features that are listed below.

Important

Technology Preview features are not supported with Red Hat production service level agreements (SLAs), might not be functionally complete, and Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information, see Red Hat Technology Preview features support scope.

  • Data Mapper supports CSV data

    For any Fuse Online connector that has an action for which you can define a data shape, you now have the option to specify a CSV instance (in addition to the options to specify JSON schema, JSON instance, XML schema, or XML instance). For example, in a Webhook connection, you can specify a CSV file as a data output type for an action.

  • Fuse Online auditing

    Fuse Online supports basic auditing for changes made by any user to the following Fuse Online components:

    • Connections - The Name and any other fields shown on the connector’s Details page in the Fuse Online web console.
    • Connectors - The Name field.
    • Integrations - The Name field.
  • Conditional expressions for mapping data fields

    In the data mapper, you can specify a conditional expression and apply it to a data mapping. For example, a conditional expression can specify evaluation of a source field and how to populate the target field if the source field is empty. The limited set of expressions that you can specify are similar to Microsoft Excel expressions.

  • Document scope for user-defined properties in data mapper

    In the data mapper, you can specify a scope for properties that you define for source and target mappings. In the Mapping Details panel, click Add (+) next to Properties. In the Create Property dialog. for the new Scope option, you can select the current message header, a message header from a previous step, or Camel Exchange Property for Camel-specific properties.

  • For a REST API client that uses OAuth, when you create an API client connector, you can change the default OAuth2 behavior of connections that you create from that connector. Fuse Online vendor extensions to the OpenAPI specification support the following:

    • Providing client credentials as parameters.
    • Obtaining a new access token based on HTTP response status codes.