Chapter 14. Configure external IP address blocks for your cluster

As a cluster administrator, you can configure the following ExternalIP settings:

  • An ExternalIP address block used by OpenShift Container Platform to automatically populate the spec.clusterIP field for a Service.
  • A policy object to restrict what IP addresses may be manually assigned to the spec.clusterIP array of a Service.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Access to the cluster as a user with the cluster-admin role.

Procedure

  1. Optional: To display the current external IP configuration, enter the following command: 

    $ oc describe networks.config cluster

  2. To edit the configuration, enter the following command: 

    $ oc edit networks.config cluster

  3. Modify the ExternalIP configuration, as in the following example: 

    apiVersion: config.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  ...
  externalIP: 1
  ...
    1
    Specify the configuration for the externalIP stanza.

  4. To confirm the updated ExternalIP configuration, enter the following command: 

    $ oc get networks.config cluster -o go-template='{{.spec.externalIP}}{{"\n"}}'

14.1. Next steps

14.2. Configuring ingress cluster traffic using an Ingress Controller

OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses an Ingress Controller.

14.2.1. Using Ingress Controllers and routes

The Ingress Operator manages Ingress Controllers and wildcard DNS.

Using an Ingress Controller is the most common way to allow external access to an OpenShift Container Platform cluster.

An Ingress Controller is configured to accept external requests and proxy them based on the configured routes. This is limited to HTTP, HTTPS using SNI, and TLS using SNI, which is sufficient for web applications and services that work over TLS with SNI.

Work with your administrator to configure an Ingress Controller to accept external requests and proxy them based on the configured routes.

The administrator can create a wildcard DNS entry and then set up an Ingress Controller. Then, you can work with the edge Ingress Controller without having to contact the administrators.

When a set of routes is created in various projects, the overall set of routes is available to the set of Ingress Controllers. Each Ingress Controller admits routes from the set of routes. By default, all Ingress Controllers admit all routes.

The Ingress Controller:

  • Has two replicas by default, which means it should be running on two worker nodes.
  • Can be scaled up to have more replicas on more nodes.
Note

The procedures in this section require prerequisites performed by the cluster administrator.

14.2.2. Prerequisites

Before starting the following procedures, the administrator must:

  • Set up the external port to the cluster networking environment so that requests can reach the cluster.

  • Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command: 

    oc adm policy add-cluster-role-to-user cluster-admin username
  • Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.

14.2.3. Creating a project and service

If the project and service that you want to expose do not exist, first create the project, then the service.

If the project and service already exist, skip to the procedure on exposing the service to create a route.

Prerequisites

  • Install the oc CLI and log in as a cluster administrator.

Procedure

  1. Create a new project for your service: 

    $ oc new-project <project_name>

    For example: 

    $ oc new-project myproject

  2. Use the oc new-app command to create a service. For example: 

    $ oc new-app \
    -e MYSQL_USER=admin \
    -e MYSQL_PASSWORD=redhat \
    -e MYSQL_DATABASE=mysqldb \
    registry.redhat.io/rhscl/mysql-80-rhel7

  3. Run the following command to see that the new service is created: 

    $ oc get svc -n myproject

    Example output

    NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
mysql-80-rhel7   ClusterIP   172.30.63.31   <none>        3306/TCP   4m55s

    By default, the new service does not have an external IP address.

14.2.4. Exposing the service by creating a route

You can expose the service as a route by using the oc expose command.

Procedure

To expose the service:

  1. Log in to OpenShift Container Platform.

  2. Log in to the project where the service you want to expose is located: 

    $ oc project project1

  3. Run the following command to expose the route: 

    $ oc expose service <service_name>

    For example: 

    $ oc expose service mysql-80-rhel7

    Example output

    route "mysql-80-rhel7" exposed

  4. Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service: 

    $ curl <pod_ip>:<port>

    For example: 

    $ curl 172.30.131.89:3306

    The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connected to the service.

    If you have a MySQL client, log in with the standard CLI command: 

    $ mysql -h 172.30.131.89 -u admin -p

    Example output

    Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.

MySQL [(none)]>

14.2.5. Configuring Ingress Controller sharding by using route labels

Ingress Controller sharding by using route labels means that the Ingress Controller serves any route in any namespace that is selected by the route selector.

Ingress Controller sharding is useful when balancing incoming traffic load among a set of Ingress Controllers and when isolating traffic to a specific Ingress Controller. For example, company A goes to one Ingress Controller and company B to another.

Procedure

  1. Edit the router-internal.yaml file: 

    # cat router-internal.yaml
apiVersion: v1
items:
- apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: sharded
    namespace: openshift-ingress-operator
  spec:
    domain: <apps-sharded.basedomain.example.net>
    nodePlacement:
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/worker: ""
    routeSelector:
      matchLabels:
        type: sharded
  status: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

  2. Apply the Ingress Controller router-internal.yaml file: 

    # oc apply -f router-internal.yaml

    The Ingress Controller selects routes in any namespace that have the label type: sharded.

14.2.6. Configuring Ingress Controller sharding by using namespace labels

Ingress Controller sharding by using namespace labels means that the Ingress Controller serves any route in any namespace that is selected by the namespace selector.

Ingress Controller sharding is useful when balancing incoming traffic load among a set of Ingress Controllers and when isolating traffic to a specific Ingress Controller. For example, company A goes to one Ingress Controller and company B to another.

Procedure

  1. Edit the router-internal.yaml file: 

    # cat router-internal.yaml

    Example output

    apiVersion: v1
items:
- apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: sharded
    namespace: openshift-ingress-operator
  spec:
    domain: <apps-sharded.basedomain.example.net>
    nodePlacement:
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/worker: ""
    namespaceSelector:
      matchLabels:
        type: sharded
  status: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

  2. Apply the Ingress Controller router-internal.yaml file: 

    # oc apply -f router-internal.yaml

    The Ingress Controller selects routes in any namespace that is selected by the namespace selector that have the label type: sharded.

14.2.7. Additional resources

14.3. Configuring ingress cluster traffic using a load balancer

OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses a load balancer.

14.3.1. Using a load balancer to get traffic into the cluster

If you do not need a specific external IP address, you can configure a load balancer service to allow external access to an OpenShift Container Platform cluster.

A load balancer service allocates a unique IP. The load balancer has a single edge router IP, which can be a virtual IP (VIP), but is still a single machine for initial load balancing.

Note

If a pool is configured, it is done at the infrastructure level, not by a cluster administrator.

Note

The procedures in this section require prerequisites performed by the cluster administrator.

14.3.2. Prerequisites

Before starting the following procedures, the administrator must:

  • Set up the external port to the cluster networking environment so that requests can reach the cluster.

  • Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command: 

    oc adm policy add-cluster-role-to-user cluster-admin username
  • Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.

14.3.3. Creating a project and service

If the project and service that you want to expose do not exist, first create the project, then the service.

If the project and service already exist, skip to the procedure on exposing the service to create a route.

Prerequisites

  • Install the oc CLI and log in as a cluster administrator.

Procedure

  1. Create a new project for your service: 

    $ oc new-project <project_name>

    For example: 

    $ oc new-project myproject

  2. Use the oc new-app command to create a service. For example: 

    $ oc new-app \
    -e MYSQL_USER=admin \
    -e MYSQL_PASSWORD=redhat \
    -e MYSQL_DATABASE=mysqldb \
    registry.redhat.io/rhscl/mysql-80-rhel7

  3. Run the following command to see that the new service is created: 

    $ oc get svc -n myproject

    Example output

    NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
mysql-80-rhel7   ClusterIP   172.30.63.31   <none>        3306/TCP   4m55s

    By default, the new service does not have an external IP address.

14.3.4. Exposing the service by creating a route

You can expose the service as a route by using the oc expose command.

Procedure

To expose the service:

  1. Log in to OpenShift Container Platform.

  2. Log in to the project where the service you want to expose is located: 

    $ oc project project1

  3. Run the following command to expose the route: 

    $ oc expose service <service_name>

    For example: 

    $ oc expose service mysql-80-rhel7

    Example output

    route "mysql-80-rhel7" exposed

  4. Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service: 

    $ curl <pod_ip>:<port>

    For example: 

    $ curl 172.30.131.89:3306

    The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connected to the service.

    If you have a MySQL client, log in with the standard CLI command: 

    $ mysql -h 172.30.131.89 -u admin -p

    Example output

    Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.

MySQL [(none)]>

14.3.5. Creating a load balancer service

Use the following procedure to create a load balancer service.

Prerequisites

  • Make sure that the project and service you want to expose exist.

Procedure

To create a load balancer service:

  1. Log in to OpenShift Container Platform.

  2. Load the project where the service you want to expose is located. 

    $ oc project project1

  3. Open a text file on the master node and paste the following text, editing the file as needed:

    Sample load balancer configuration file

    apiVersion: v1
kind: Service
metadata:
  name: egress-2 1
spec:
  ports:
  - name: db
    port: 3306 2
  loadBalancerIP:
  type: LoadBalancer 3
  selector:
    name: mysql 4

    1
    Enter a descriptive name for the load balancer service.
    2
    Enter the same port that the service you want to expose is listening on.
    3
    Enter loadbalancer as the type.
    4
    Enter the name of the service.
  4. Save and exit the file.

  5. Run the following command to create the service: 

    $ oc create -f <file-name>

    For example: 

    $ oc create -f mysql-lb.yaml

  6. Execute the following command to view the new service: 

    $ oc get svc

    Example output

    NAME       TYPE           CLUSTER-IP      EXTERNAL-IP                             PORT(S)          AGE
egress-2   LoadBalancer   172.30.22.226   ad42f5d8b303045-487804948.example.com   3306:30357/TCP   15m

    The service has an external IP address automatically assigned if there is a cloud provider enabled.

  7. On the master, use a tool, such as cURL, to make sure you can reach the service using the public IP address: 

    $ curl <public-ip>:<port>

    For example: 

    $ curl 172.29.121.74:3306

    The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connecting with the service:

    If you have a MySQL client, log in with the standard CLI command: 

    $ mysql -h 172.30.131.89 -u admin -p

    Example output

    Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.

MySQL [(none)]>

14.4. Configuring ingress cluster traffic for a service external IP

You can attach an external IP address to a Service so that it is available to traffic outside the cluster. This is generally useful only for a cluster installed on bare metal hardware. The external network infrastructure must be configured correctly to route traffic to the Service.

14.4.1. Prerequisites

14.4.2. Attaching an ExternalIP to a Service

You can attach an ExternalIP to a service. If your cluster is configured to allocate an ExternalIP automatically, you might not need to manually attach an ExternalIP to the Service.

Procedure

  1. Optional: To confirm what IP address ranges are configured for use with ExternalIP, enter the following command: 

    $ oc get networks.config cluster -o jsonpath='{.spec.externalIP}{"\n"}'

    If autoAssignCIDRs is set, OpenShift Container Platform automatically assigns an ExternalIP to a new Service if the spec.externalIPs field is not specified.

  2. Attach an ExternalIP to the Service resource.

    1. If you are creating a new Service, specify the spec.externalIPs field and provide an array of one or more valid IP addresses. For example: 

      apiVersion: v1
kind: Service
metadata:
  name: svc-with-externalip
spec:
  ...
  externalIPs:
  - 192.174.120.10

    2. If you are attaching an ExternalIP to an existing Service, enter the following command. Replace <name> with the Service name. Replace <ip_address> with a valid ExternalIP address. You can provide multiple IP addresses separated by commas. 

      $ oc patch svc <name> -p \
  '{
    "spec": {
      "externalIPs": [ "<ip_address>" ]
    }
  }'

      For example: 

      $ oc patch svc mysql-55-rhel7 -p '{"spec":{"externalIPs":["192.174.120.10"]}}'

      Example output

      "mysql-55-rhel7" patched

  3. To confirm that an ExternalIP address is attached to the Service, enter the following command. If you specified an ExternalIP for a new Service, you must create the Service first. 

    $ oc get svc

    Example output

    NAME               CLUSTER-IP      EXTERNAL-IP     PORT(S)    AGE
mysql-55-rhel7     172.30.131.89   192.174.120.10  3306/TCP   13m

14.4.3. Additional resources

14.5. Configuring ingress cluster traffic using a NodePort

OpenShift Container Platform provides methods for communicating from outside the cluster with services running in the cluster. This method uses a NodePort.

14.5.1. Using a NodePort to get traffic into the cluster

Use a NodePort-type Service resource to expose a service on a specific port on all nodes in the cluster. The port is specified in the Service resource’s .spec.ports[*].nodePort field.

Important

Using NodePorts requires additional port resources.

A NodePort exposes the service on a static port on the node’s IP address. NodePorts are in the 30000 to 32767 range by default, which means a NodePort is unlikely to match a service’s intended port. For example, port 8080 may be exposed as port 31020 on the node.

The administrator must ensure the external IP addresses are routed to the nodes.

NodePorts and external IPs are independent and both can be used concurrently.

Note

The procedures in this section require prerequisites performed by the cluster administrator.

14.5.2. Prerequisites

Before starting the following procedures, the administrator must:

  • Set up the external port to the cluster networking environment so that requests can reach the cluster.

  • Make sure there is at least one user with cluster admin role. To add this role to a user, run the following command: 

    $ oc adm policy add-cluster-role-to-user cluster-admin <user_name>
  • Have an OpenShift Container Platform cluster with at least one master and at least one node and a system outside the cluster that has network access to the cluster. This procedure assumes that the external system is on the same subnet as the cluster. The additional networking required for external systems on a different subnet is out-of-scope for this topic.

14.5.3. Creating a project and service

If the project and service that you want to expose do not exist, first create the project, then the service.

If the project and service already exist, skip to the procedure on exposing the service to create a route.

Prerequisites

  • Install the oc CLI and log in as a cluster administrator.

Procedure

  1. Create a new project for your service: 

    $ oc new-project <project_name>

    For example: 

    $ oc new-project myproject

  2. Use the oc new-app command to create a service. For example: 

    $ oc new-app \
    -e MYSQL_USER=admin \
    -e MYSQL_PASSWORD=redhat \
    -e MYSQL_DATABASE=mysqldb \
    registry.redhat.io/rhscl/mysql-80-rhel7

  3. Run the following command to see that the new service is created: 

    $ oc get svc -n myproject

    Example output

    NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
mysql-80-rhel7   ClusterIP   172.30.63.31   <none>        3306/TCP   4m55s

    By default, the new service does not have an external IP address.

14.5.4. Exposing the service by creating a route

You can expose the service as a route by using the oc expose command.

Procedure

To expose the service:

  1. Log in to OpenShift Container Platform.

  2. Log in to the project where the service you want to expose is located: 

    $ oc project project1

  3. To expose a node port for the application, enter the following command. OpenShift Container Platform automatically selects an available port in the 30000-32767 range. 

    $ oc expose dc mysql-80-rhel7 --type=NodePort --name=mysql-ingress

  4. Optional: To confirm the service is available with a node port exposed, enter the following command: 

    $ oc get svc -n myproject

    Example output

    NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
mysql-80-rhel7   ClusterIP   172.30.217.127   <none>        3306/TCP         9m44s
mysql-ingress    NodePort    172.30.107.72    <none>        3306:31345/TCP   39s

  5. Optional: To remove the service created automatically by the oc new-app command, enter the following command: 

    $ oc delete svc mysql-80-rhel7