Chapter 5. Creating a router network

To create a network of AMQ Interconnect routers, you define a deployment in an Interconnect Custom Resource, and then apply it. The AMQ Interconnect Operator creates the deployment by scheduling the necessary Pods and creating any needed Resources.

The procedures in this section demonstrate the following router network topologies:

  • Interior router mesh
  • Interior router mesh with edge routers for scalability
  • Inter-cluster router network that connects two OpenShift clusters

Prerequisites

  • The AMQ Interconnect Operator is installed in your OpenShift Container Platform project.

5.1. Creating an interior router deployment

Interior routers establish connections with each other and automatically compute the lowest cost paths across the network.

Procedure

This procedure creates an interior router network of three routers. The routers automatically connect to each other in a mesh topology, and their connections are secured with mutual SSL/TLS authentication.

  1. Create an Interconnect Custom Resource YAML file that describes the interior router deployment.

    Sample router-mesh.yaml file

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: router-mesh
spec:
  deploymentPlan:
    role: interior  1
    size: 3  2
    placement:  Any  3

    1
    The operating mode of the routers in the deployment. The Operator will automatically connect interior routers in a mesh topology.
    2
    The number of routers to create.
    3
    Each router runs in a separate Pod. The placement defines where in the cluster the Operator should schedule and place the Pods. You can choose the following placement options:
    Any
    The Pods can run on any node in the OpenShift Container Platform cluster.
    Every
    The Operator places a router Pod on each node in the cluster. If you choose this option, the Size property is not needed - the number of routers corresponds to the number of nodes in the cluster.
    AntiAffinity
    The Operator ensures that multiple router Pods do not run on the same node in the cluster. If the size is greater than the number of nodes in the cluster, the extra Pods that cannot be scheduled will remain in a Pending state.

  2. Create the router deployment described in the YAML file. 

    $ oc apply -f router-mesh.yaml

    The Operator creates a deployment of interior routers in a mesh topology that uses default address semantics. It also creates a Service through which the routers can be accessed, and a Route through which you can access the web console.

  3. Verify that the router mesh was created and the Pods are running.

    Each router runs in a separate Pod. They connect to each other automatically using the Service that the Operator created. 

    $ oc get pods
NAME                                     READY   STATUS    RESTARTS   AGE
interconnect-operator-587f94784b-4bzdx   1/1     Running   0          52m
router-mesh-6b48f89bd-588r5              1/1     Running   0          40m
router-mesh-6b48f89bd-bdjc4              1/1     Running   0          40m
router-mesh-6b48f89bd-h6d5r              1/1     Running   0          40m

  4. Review the router deployment. 

    $ oc get interconnect/router-mesh -o yaml
apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
...
spec:
  addresses:  1
  - distribution: closest
    prefix: closest
  - distribution: multicast
    prefix: multicast
  - distribution: closest
    prefix: unicast
  - distribution: closest
    prefix: exclusive
  - distribution: multicast
    prefix: broadcast
  deploymentPlan:  2
    livenessPort: 8888
    placement: Any
    resources: {}
    role: interior
    size: 3
  edgeListeners:  3
  - port: 45672
  interRouterListeners:  4
  - authenticatePeer: true
    expose: true
    port: 55671
    saslMechanisms: EXTERNAL
    sslProfile: inter-router
  listeners:  5
  - port: 5672
  - authenticatePeer: true
    expose: true
    http: true
    port: 8080
  - port: 5671
    sslProfile: default
  sslProfiles:  6
  - credentials: router-mesh-default-tls
    name: default
  - caCert: router-mesh-inter-router-tls
    credentials: router-mesh-inter-router-tls
    mutualAuth: true
    name: inter-router
  users: router-mesh-users  7
    1
    The default address configuration. All messages sent to an address that does not match any of these prefixes are distributed in a balanced anycast pattern.
    2
    A router mesh of three interior routers was deployed.
    3
    Each interior router listens on port 45672 for connections from edge routers.
    4
    The interior routers connect to each other on port 55671. These inter-router connections are secured with SSL/TLS mutual authentication. The inter-router SSL Profile contains the details of the certificates that the Operator generated.
    5
    Each interior router listens for connections from external clients on the following ports:
    • 5672 - Unsecure connections from messaging applications.
    • 5671 - Secure connections from messaging applications.
    • 8080 - AMQ Interconnect web console access. Default user name/password security is applied.
    6
    Using the Red Hat Integration - AMQ Certificate Manager Operator, the Red Hat Integration - AMQ Interconnect Operator automatically creates two SSL profiles:
    • inter-router - The Operator secures the inter-router network with mutual TLS authentication by creating a Certificate Authority (CA) and generating certificates signed by the CA for each interior router.
    • default - The Operator creates TLS certificates for messaging applications to connect to the interior routers on port 5671.
    7
    The AMQ Interconnect web console is secured with user name/password authentication. The Operator automatically generates the credentials and stores them in the router-mesh-users Secret.

5.2. Creating an edge router deployment

You can efficiently scale your router network by adding an edge router deployment. Edge routers act as connection concentrators for messaging applications. Each edge router maintains a single uplink connection to an interior router, and messaging applications connect to the edge routers to send and receive messages.

Prerequisites

Procedure

This procedure creates an edge router on each node of the OpenShift Container Platform cluster and connects them to the previously created interior router mesh.

  1. Create an Interconnect Custom Resource YAML file that describes the edge router deployment.

    Sample edge-routers.yaml file

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: edge-routers
spec:
  deploymentPlan:
    role: edge
    placement: Every  1
  edgeConnectors:  2
    - host: router-mesh  3
      port: 45672  4

    1
    An edge router Pod will be deployed on each node in the OpenShift Container Platform cluster. This placement helps to balance messaging application traffic across the cluster. The Operator will create a DaemonSet to ensure that the number of Pods scheduled always corresponds to the number of nodes in the cluster.
    2
    Edge connectors define the connections from the edge routers to the interior routers.
    3
    The name of the Service that was created for the interior routers.
    4
    The port on which the interior routers listen for edge connections. The default is 45672.

  2. Create the edge routers described in the YAML file: 

    $ oc apply -f edge-routers.yaml

    The Operator deploys an edge router on each node of the OpenShift Container Platform cluster, and connects them to the interior routers.

  3. Verify that the edge routers were created and the Pods are running.

    Each router runs in a separate Pod. Each edge router connects to any of the previously created interior routers. 

    $ oc get pods
NAME                                     READY   STATUS    RESTARTS   AGE
edge-routers-2jz5j                       1/1     Running   0          33s
edge-routers-fhlxv                       1/1     Running   0          33s
edge-routers-gg2qb                       1/1     Running   0          33s
edge-routers-hj72t                       1/1     Running   0          33s
interconnect-operator-587f94784b-4bzdx   1/1     Running   0          54m
router-mesh-6b48f89bd-588r5              1/1     Running   0          42m
router-mesh-6b48f89bd-bdjc4              1/1     Running   0          42m
router-mesh-6b48f89bd-h6d5r              1/1     Running   0          42m

5.3. Creating an inter-cluster router network

Depending on whether you are using AMQ Certificate Manager, there are different procedures for creating an inter-cluster router network.

5.3.1. Creating an inter-cluster router network using a Certificate Authority

You can create a router network from routers running in different OpenShift Container Platform clusters. This enables you to connect applications running in separate clusters.

Prerequisites

  • You have already created secrets defining an existing certificate for each router.

Procedure

This procedure creates router deployments in two different OpenShift Container Platform clusters (cluster1 and cluster2) and connects them together to form an inter-cluster router network. The connection between the router deployments is secured with SSL/TLS mutual authentication.

  1. In the first OpenShift Container Platform cluster (cluster1), create an Interconnect Custom Resource YAML file that describes the interior router deployment.

    This example creates a single interior router with a default configuration.

    Sample cluster1-router-mesh.yaml file

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: cluster1-router-mesh
spec:
  interRouterListeners:
    - authenticatePeer: true 1
      host: 0.0.0.0 2
      port: 55672 3
      saslMechanisms: EXTERNAL 4
      sslProfile: inter-router-profile 5
      expose: true 6
  sslProfiles:
    - caCert: inter-router-certs-secret 7
      credentials: inter-router-certs-secret 8
      name: inter-router-profile 9

    1
    authenticatePeer must be set to true to authenticate using TLS certificates
    2
    listener host
    3
    listener port
    4
    SASL mechanism to authenticate, use EXTERNAL for TLS certificates
    5
    ssl-profile name to use for authenticating clients
    6
    exposes a route so that the port is accessible from outside the cluster
    7
    name of cluster secret or your CA containing a ca.crt name (in case you’re using the same secret used in credentials, otherwise it must have a tls.crt)
    8
    name of cluster secret with the CA certificate containing tls.crt and tls.key files
    9
    ssl-profile name to use for the interRouterListener

  2. Create the router deployment described in the YAML file. 

    $ oc apply -f cluster1-router-mesh.yaml

    The Red Hat Integration - AMQ Interconnect Operator creates an interior router with a default configuration and a listener to authenticate other routers.

  3. Log in to the second OpenShift Container Platform cluster (cluster2), and switch to the project where you want to create the second router deployment.

  4. In cluster2, create an Interconnect Custom Resource YAML file to describe the router deployment. 

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: cluster2-router-mesh
spec:
  sslProfiles:
  - name: inter-router-profile  1
    credentials: inter-router-certs-secret
    caCert: inter-router-certs-secret
  interRouterConnectors:
  - host: cluster1-router-mesh-port-55672-myproject.cluster1.openshift.com  2
    port: 443
    verifyHostname: false
    sslProfile: inter-router-profile
    name: cluster1
    1
    This SSL Profile defines the certificate needed to connect to the router deployment in cluster1.
    2
    The URL of the Route for the inter-router listener on cluster1.

  5. Create the router deployment described in the YAML file. 

    $ oc apply -f cluster2-router-mesh.yaml

  6. Verify that the routers are connected.

    This example displays the connections from the router in cluster2 to the router in cluster1. 

    $ oc exec cluster2-fb6bc5797-crvb6 -it -- qdstat -c
Connections
  id    host                                                                  container                              role          dir  security                                authentication  tenant
  ====================================================================================================================================================================================================
  1     cluster1-router-mesh-port-55672-myproject.cluster1.openshift.com:443  cluster1-router-mesh-54cffd9967-9h4vq  inter-router  out  TLSv1/SSLv3(DHE-RSA-AES256-GCM-SHA384)  x.509

5.3.2. Creating an inter-cluster router network using AMQ Certificate Manager

You can create a router network from routers running in different OpenShift Container Platform clusters. This enables you to connect applications running in separate clusters.

Procedure

This procedure creates router deployments in two different OpenShift Container Platform clusters (cluster1 and cluster2) and connects them together to form an inter-cluster router network. The connection between the router deployments is secured with SSL/TLS mutual authentication.

  1. In the first OpenShift Container Platform cluster (cluster1), create an Interconnect Custom Resource YAML file that describes the interior router deployment.

    This example creates a single interior router with a default configuration.

    Sample cluster1-router-mesh.yaml file

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: cluster1-router-mesh
spec: {}

  2. Create the router deployment described in the YAML file. 

    $ oc apply -f cluster1-router-mesh.yaml

    The Red Hat Integration - AMQ Interconnect Operator creates an interior router with a default configuration. It uses the Red Hat Integration - AMQ Certificate Manager Operator to create a Certificate Authority (CA) and generate a certificate signed by the CA.

  3. Generate an additional certificate for the router deployment in the second OpenShift Container Platform cluster (cluster2).

    The router deployment in cluster2 requires a certificate issued by the CA of cluster1.

    1. Create a Certificate Custom Resource YAML file to request a certificate.

      Sample certificate-request.yaml file

      apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: cluster2-inter-router-tls
spec:
  commonName: cluster1-router-mesh-myproject.cluster2.openshift.com
  issuerRef:
    name: cluster1-router-mesh-inter-router-ca  1
  secretName: cluster2-inter-router-tls-secret
---

      1
      The name of the Issuer that created the inter-router CA for cluster1. By default, the name of the Issuer is <application-name>-inter-router-ca.

    2. Create the certificate described in the YAML file. 

      $ oc apply -f certificate-request.yaml

    3. Extract the certificate that you generated. 

      $ mkdir /tmp/cluster2-inter-router-tls
$ oc extract secret/cluster2-inter-router-tls-secret --to=/tmp/cluster2-inter-router-tls
  4. Log in to the second OpenShift Container Platform cluster (cluster2), and switch to the project where you want to create the second router deployment.

  5. In cluster2, create a Secret containing the certificate that you generated. 

    $ oc create secret generic cluster2-inter-router-tls-secret --from-file=/tmp/cluster2-inter-router-tls

  6. In cluster2, create an Interconnect Custom Resource YAML file to describe the router deployment. 

    apiVersion: interconnectedcloud.github.io/v1alpha1
kind: Interconnect
metadata:
  name: cluster2-router-mesh
spec:
  sslProfiles:
  - name: inter-cluster-tls  1
    credentials: cluster2-inter-router-tls-secret
    caCert: cluster2-inter-router-tls-secret
  interRouterConnectors:
  - host: cluster1-router-mesh-port-55671-myproject.cluster1.openshift.com  2
    port: 443
    verifyHostname: false
    sslProfile: inter-cluster-tls
    1
    This SSL Profile defines the certificate needed to connect to the router deployment in cluster1.
    2
    The URL of the Route for the inter-router listener on cluster1.

  7. Create the router deployment described in the YAML file. 

    $ oc apply -f cluster2-router-mesh.yaml

  8. Verify that the routers are connected.

    This example displays the connections from the router in cluster2 to the router in cluster1. 

    $ oc exec cluster2-fb6bc5797-crvb6 -it -- qdstat -c
Connections
  id    host                                                                  container                              role          dir  security                                authentication  tenant
  ====================================================================================================================================================================================================
  1     cluster1-router-mesh-port-55671-myproject.cluster1.openshift.com:443  cluster1-router-mesh-54cffd9967-9h4vq  inter-router  out  TLSv1/SSLv3(DHE-RSA-AES256-GCM-SHA384)  x.509