Menu Close

Chapter 10. Multiple networks

10.1. Understanding multiple networks

In Kubernetes, container networking is delegated to networking plug-ins that implement the Container Network Interface (CNI).

OpenShift Container Platform uses the Multus CNI plug-in to allow chaining of CNI plug-ins. During cluster installation, you configure your default pod network. The default network handles all ordinary network traffic for the cluster. You can define an additional network based on the available CNI plug-ins and attach one or more of these networks to your pods. You can define more than one additional network for your cluster, depending on your needs. This gives you flexibility when you configure pods that deliver network functionality, such as switching or routing.

10.1.1. Usage scenarios for an additional network

You can use an additional network in situations where network isolation is needed, including data plane and control plane separation. Isolating network traffic is useful for the following performance and security reasons:

Performance
You can send traffic on two different planes in order to manage how much traffic is along each plane.
Security
You can send sensitive traffic onto a network plane that is managed specifically for security considerations, and you can separate private data that must not be shared between tenants or customers.

All of the pods in the cluster still use the cluster-wide default network to maintain connectivity across the cluster. Every pod has an eth0 interface that is attached to the cluster-wide pod network. You can view the interfaces for a pod by using the oc exec -it <pod_name> -- ip a command. If you add additional network interfaces that use Multus CNI, they are named net1, net2, …​, netN.

To attach additional network interfaces to a pod, you must create configurations that define how the interfaces are attached. You specify each interface by using a NetworkAttachmentDefinition custom resource (CR). A CNI configuration inside each of these CRs defines how that interface is created.

10.1.2. Additional networks in OpenShift Container Platform

OpenShift Container Platform provides the following CNI plug-ins for creating additional networks in your cluster:

10.2. Attaching a pod to an additional network

As a cluster user you can attach a pod to an additional network.

10.2.1. Adding a pod to an additional network

You can add a pod to an additional network. The pod continues to send normal cluster-related network traffic over the default network.

When a pod is created additional networks are attached to it. However, if a pod already exists, you cannot attach additional networks to it.

The pod must be in the same namespace as the additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in to the cluster.

Procedure

  1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

    1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1
      1
      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.
    2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]
      1
      Specify the name of the additional network defined by a NetworkAttachmentDefinition object.
      2
      Specify the namespace where the NetworkAttachmentDefinition object is defined.
      3
      Optional: Specify an override for the default route, such as 192.168.17.1.
  2. To create the pod, enter the following command. Replace <name> with the name of the pod.

    $ oc create -f <name>.yaml
  3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the pod.

    $ oc get pod <name> -o yaml

    In the following example, the example-pod pod is attached to the net1 additional network:

    $ oc get pod example-pod -o yaml
    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        k8s.v1.cni.cncf.io/networks: macvlan-bridge
        k8s.v1.cni.cncf.io/networks-status: |- 1
          [{
              "name": "openshift-sdn",
              "interface": "eth0",
              "ips": [
                  "10.128.2.14"
              ],
              "default": true,
              "dns": {}
          },{
              "name": "macvlan-bridge",
              "interface": "net1",
              "ips": [
                  "20.2.2.100"
              ],
              "mac": "22:2f:60:a5:f8:00",
              "dns": {}
          }]
      name: example-pod
      namespace: default
    spec:
      ...
    status:
      ...
    1
    The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.

10.2.1.1. Specifying pod-specific addressing and routing options

When attaching a pod to an additional network, you may want to specify further properties about that network in a particular pod. This allows you to change some aspects of routing, as well as specify static IP addresses and MAC addresses. In order to accomplish this, you can use the JSON formatted annotations.

Prerequisites

  • The pod must be in the same namespace as the additional network.
  • Install the OpenShift Command-line Interface (oc).
  • You must log in to the cluster.

Procedure

To add a pod to an additional network while specifying addressing and/or routing options, complete the following steps:

  1. Edit the Pod resource definition. If you are editing an existing Pod resource, run the following command to edit its definition in the default editor. Replace <name> with the name of the Pod resource to edit.

    $ oc edit pod <name>
  2. In the Pod resource definition, add the k8s.v1.cni.cncf.io/networks parameter to the pod metadata mapping. The k8s.v1.cni.cncf.io/networks accepts a JSON string of a list of objects that reference the name of NetworkAttachmentDefinition custom resource (CR) names in addition to specifying additional properties.

    metadata:
      annotations:
        k8s.v1.cni.cncf.io/networks: '[<network>[,<network>,...]]' 1
    1
    Replace <network> with a JSON object as shown in the following examples. The single quotes are required.
  3. In the following example the annotation specifies which network attachment will have the default route, using the default-route parameter.

    apiVersion: v1
    kind: Pod
    metadata:
      name: example-pod
      annotations:
        k8s.v1.cni.cncf.io/networks: '
        {
          "name": "net1"
        },
        {
          "name": "net2", 1
          "default-route": ["192.0.2.1"] 2
        }'
    spec:
      containers:
      - name: example-pod
        command: ["/bin/bash", "-c", "sleep 2000000000000"]
        image: centos/tools
    1
    The name key is the name of the additional network to associate with the pod.
    2
    The default-route key specifies a value of a gateway for traffic to be routed over if no other routing entry is present in the routing table. If more than one default-route key is specified, this will cause the pod to fail to become active.

The default route will cause any traffic that is not specified in other routes to be routed to the gateway.

Important

Setting the default route to an interface other than the default network interface for OpenShift Container Platform may cause traffic that is anticipated for pod-to-pod traffic to be routed over another interface.

To verify the routing properties of a pod, the oc command may be used to execute the ip command within a pod.

$ oc exec -it <pod_name> -- ip route
Note

You may also reference the pod’s k8s.v1.cni.cncf.io/networks-status to see which additional network has been assigned the default route, by the presence of the default-route key in the JSON-formatted list of objects.

To set a static IP address or MAC address for a pod you can use the JSON formatted annotations. This requires you create networks that specifically allow for this functionality. This can be specified in a rawCNIConfig for the CNO.

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration

name: <name> 1
namespace: <namespace> 2
rawCNIConfig: '{ 3
  ...
}'
type: Raw

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If you do not specify a value, then the default namespace is used.
3
Specify the CNI plug-in configuration in JSON format, which is based on the following template.

The following object describes the configuration parameters for utilizing static MAC address and IP address using the macvlan CNI plug-in:

macvlan CNI plug-in JSON configuration object using static IP and MAC address

{
  "cniVersion": "0.3.1",
  "name": "<name>", 1
  "plugins": [{ 2
      "type": "macvlan",
      "capabilities": { "ips": true }, 3
      "master": "eth0", 4
      "mode": "bridge",
      "ipam": {
        "type": "static"
      }
    }, {
      "capabilities": { "mac": true }, 5
      "type": "tuning"
    }]
}

1
Specifies the name for the additional network attachment to create. The name must be unique within the specified namespace.
2
Specifies an array of CNI plug-in configurations. The first object specifies a macvlan plug-in configuration and the second object specifies a tuning plug-in configuration.
3
Specifies that a request is made to enable the static IP address functionality of the CNI plug-in runtime configuration capabilities.
4
Specifies the interface that the macvlan plug-in uses.
5
Specifies that a request is made to enable the static MAC address functionality of a CNI plug-in.

The above network attachment can be referenced in a JSON formatted annotation, along with keys to specify which static IP and MAC address will be assigned to a given pod.

Edit the pod with:

$ oc edit pod <name>

macvlan CNI plug-in JSON configuration object using static IP and MAC address

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
      {
        "name": "<name>", 1
        "ips": [ "192.0.2.205/24" ], 2
        "mac": "CA:FE:C0:FF:EE:00" 3
      }
    ]'

1
Use the <name> as provided when creating the rawCNIConfig above.
2
Provide an IP address including the subnet mask.
3
Provide the MAC address.
Note

Static IP addresses and MAC addresses do not have to be used at the same time, you may use them individually, or together.

To verify the IP address and MAC properties of a pod with additional networks, use the oc command to execute the ip command within a pod.

$ oc exec -it <pod_name> -- ip a

10.3. Removing a pod from an additional network

As a cluster user you can remove a pod from an additional network.

10.3.1. Removing a pod from an additional network

You can remove a pod from an additional network only by deleting the pod.

Prerequisites

  • An additional network is attached to the pod.
  • Install the OpenShift CLI (oc).
  • Log in to the cluster.

Procedure

  • To delete the pod, enter the following command:

    $ oc delete pod <name> -n <namespace>
    • <name> is the name of the pod.
    • <namespace> is the namespace that contains the pod.

10.4. Configuring a bridge network

As a cluster administrator, you can configure an additional network for your cluster using the bridge Container Network Interface (CNI) plug-in. When configured, all Pods on a node are connected to a virtual switch. Each pod is assigned an IP address on the additional network.

10.4.1. Creating an additional network attachment with the bridge CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition object automatically.

Important

Do not edit the NetworkAttachmentDefinition objects that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the bridge CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: 1
      - name: test-network-1
        namespace: test-1
        type: Raw
        rawCNIConfig: '{
          "cniVersion": "0.3.1",
          "name": "test-network-1",
          "type": "bridge",
          "ipam": {
            "type": "static",
            "addresses": [
              {
                "address": "192.168.1.23/24"
              }
            ]
          }
        }'
    1
    Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.
  4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

    $ oc get network-attachment-definitions -n <namespace>

    Example output

    NAME                 AGE
    test-network-1       14m

10.4.1.1. Configuration for bridge

The configuration for an additional network attachment that uses the bridge Container Network Interface (CNI) plug-in is provided in two parts:

  • Cluster Network Operator (CNO) configuration
  • CNI plug-in configuration

The CNO configuration specifies the name for the additional network attachment and the namespace to create the attachment in. The plug-in is configured by a JSON object specified by the rawCNIConfig parameter in the CNO configuration.

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration

name: <name> 1
namespace: <namespace> 2
rawCNIConfig: '{ 3
  ...
}'
type: Raw

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If you do not specify a value, then the default namespace is used.
3
Specify the CNI plug-in configuration in JSON format, which is based on the following template.

The following object describes the configuration parameters for the bridge CNI plug-in:

bridge CNI plug-in JSON configuration object

{
  "cniVersion": "0.3.1",
  "name": "<name>", 1
  "type": "bridge",
  "bridge": "<bridge>", 2
  "ipam": { 3
    ...
  },
  "ipMasq": false, 4
  "isGateway": false, 5
  "isDefaultGateway": false, 6
  "forceAddress": false, 7
  "hairpinMode": false, 8
  "promiscMode": false, 9
  "vlan": <vlan>, 10
  "mtu": <mtu> 11
}

1
Specify the value for the name parameter you provided previously for the CNO configuration.
2
Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, it is created. The default value is cni0.
3
Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the network attachment definition.
4
Set to true to enable IP masquerading for traffic that leaves the virtual network. The source IP address for all traffic is rewritten to the bridge’s IP address. If the bridge does not have an IP address, this setting has no effect. The default value is false.
5
Set to true to assign an IP address to the bridge. The default value is false.
6
Set to true to configure the bridge as the default gateway for the virtual network. The default value is false. If isDefaultGateway is set to true, then isGateway is also set to true automatically.
7
Set to true to allow assignment of a previously assigned IP address to the virtual bridge. When set to false, if an IPv4 address or an IPv6 address from overlapping subsets is assigned to the virtual bridge, an error occurs. The default value is false.
8
Set to true to allow the virtual bridge to send an ethernet frame back through the virtual port it was received on. This mode is also known as reflective relay. The default value is false.
9
Set to true to enable promiscuous mode on the bridge. The default value is false.
10
Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned.
11
Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
10.4.1.1.1. bridge configuration example

The following example configures an additional network named bridge-net:

name: bridge-net
namespace: work-network
type: Raw
rawCNIConfig: '{ 1
  "cniVersion": "0.3.1",
  "name": "work-network",
  "type": "bridge",
  "isGateway": true,
  "vlan": 2,
  "ipam": {
    "type": "dhcp"
    }
}'
1
The CNI configuration object is specified as a YAML string.

10.4.1.2. Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

You can use the following methods for IP address assignment:

  • Static assignment.
  • Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.
  • Dynamic assignment through the Whereabouts IPAM CNI plug-in.
10.4.1.2.1. Static IP address assignment configuration

The following JSON describes the configuration for static IP address assignment:

Static assignment configuration

{
  "ipam": {
    "type": "static",
    "addresses": [ 1
      {
        "address": "<address>", 2
        "gateway": "<gateway>" 3
      }
    ],
    "routes": [ 4
      {
        "dst": "<dst>", 5
        "gw": "<gw>" 6
      }
    ],
    "dns": { 7
      "nameservers": ["<nameserver>"], 8
      "domain": "<domain>", 9
      "search": ["<search_domain>"] 10
    }
  }
}

1
An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2
An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3
The default gateway to route egress network traffic to.
4
An array describing routes to configure inside the pod.
5
The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6
The gateway where network traffic is routed.
7
Optional: DNS configuration.
8
An of array of one or more IP addresses for to send DNS queries to.
9
The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10
An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.
10.4.1.2.2. Dynamic IP address assignment configuration

The following JSON describes the configuration for dynamic IP address address assignment with DHCP.

Renewal of DHCP leases

A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.

To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:

Example shim network attachment definition

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  ...
  additionalNetworks:
  - name: dhcp-shim
    namespace: default
    type: Raw
    rawCNIConfig: |-
      {
        "name": "dhcp-shim",
        "cniVersion": "0.3.1",
        "type": "bridge",
        "ipam": {
          "type": "dhcp"
        }
      }

DHCP assignment configuration

{
  "ipam": {
    "type": "dhcp"
  }
}

10.4.1.2.3. Dynamic IP address assignment configuration with Whereabouts

The Whereabouts CNI plug-in allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.

The following JSON describes the configuration for dynamic IP address assignment with Whereabouts:

Whereabouts assignment configuration

{
  "ipam": {
    "type": "whereabouts",
    "range": "<range>", 1
    "exclude": ["<exclude_part>, ..."], 2
  }
}

1
Specify an IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
2
Optional: Specify a list of IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
10.4.1.2.4. Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.7"
        }
      ]
  }
}
10.4.1.2.5. Dynamic IP address assignment configuration example using DHCP

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "dhcp"
  }
}
10.4.1.2.6. Dynamic IP address assignment configuration example using Whereabouts

You can configure ipam to use Whereabouts:

{
  "ipam": {
    "type": "whereabouts",
    "range": "192.0.2.192/27",
    "exclude": [
       "192.0.2.192/30",
       "192.0.2.196/32"
    ]
  }
}

10.4.2. Next steps

10.5. Configuring a host-device network

As a cluster administrator, you can configure an additional network for your cluster by using the host-device Container Network Interface (CNI) plug-in. The plug-in moves the specified network device from the network namespace of the host into the network namespace of the pod.

10.5.1. Creating an additional network attachment with the host-device CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition object automatically.

Important

Do not edit the NetworkAttachmentDefinition objects that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the host-device CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: 1
      - name: test-network-1
        namespace: test-1
        type: Raw
        rawCNIConfig: '{
          "cniVersion": "0.3.1",
          "name": "test-network-1",
          "type": "host-device",
          "device": "eth1",
          "ipam": {
            "type": "static",
            "addresses": [
              {
                "address": "192.168.1.23/24"
              }
            ]
          }
        }'
    1
    Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.
  4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

    $ oc get network-attachment-definitions -n <namespace>

    Example output

    NAME                 AGE
    test-network-1       14m

10.5.1.1. Configuration for host-device

The configuration for an additional network attachment that uses the host-device Container Network Interface (CNI) plug-in is provided in two parts:

  • Cluster Network Operator (CNO) configuration
  • CNI plug-in configuration

The CNO configuration specifies the name for the additional network attachment and the namespace to create the attachment in. The plug-in is configured by a JSON object specified by the rawCNIConfig parameter in the CNO configuration.

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration

name: <name> 1
namespace: <namespace> 2
rawCNIConfig: '{ 3
  ...
}'
type: Raw

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If you do not specify a value, the default namespace is used.
3
Specify the CNI plug-in configuration in JSON format, which is based on the following template.
Important

Specify your network device by setting only one of the following parameters: device, hwaddr, kernelpath, or pciBusID.

The following object describes the configuration parameters for the host-device CNI plug-in:

host-device CNI plug-in JSON configuration object

{
  "cniVersion": "0.3.1",
  "name": "<name>", 1
  "type": "host-device",
  "device": "<device>", 2
  "hwaddr": "<hwaddr>", 3
  "kernelpath": "<kernelpath>", 4
  "pciBusID": "<pciBusID>", 5
  "ipam": { 6
    ...
  }
}

1
Specify the value for the name parameter you provided previously for the CNO configuration.
2
Specify the name of the device, such as eth0.
3
Specify the device hardware MAC address.
4
Specify the Linux kernel device path, such as /sys/devices/pci0000:00/0000:00:1f.6.
5
Specify the PCI address of the network device, such as 0000:00:1f.6.
6
Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition.
10.5.1.1.1. host-device configuration example

The following example configures an additional network named hostdev-net:

name: hostdev-net
namespace: work-network
type: Raw
rawCNIConfig: '{ 1
  "cniVersion": "0.3.1",
  "name": "work-network",
  "type": "host-device",
  "device": "eth1",
  "ipam": {
    "type": "dhcp"
  }
}'
1
The CNI configuration object is specified as a YAML string.

10.5.1.2. Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

You can use the following methods for IP address assignment:

  • Static assignment.
  • Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.
  • Dynamic assignment through the Whereabouts IPAM CNI plug-in.
10.5.1.2.1. Static IP address assignment configuration

The following JSON describes the configuration for static IP address assignment:

Static assignment configuration

{
  "ipam": {
    "type": "static",
    "addresses": [ 1
      {
        "address": "<address>", 2
        "gateway": "<gateway>" 3
      }
    ],
    "routes": [ 4
      {
        "dst": "<dst>", 5
        "gw": "<gw>" 6
      }
    ],
    "dns": { 7
      "nameservers": ["<nameserver>"], 8
      "domain": "<domain>", 9
      "search": ["<search_domain>"] 10
    }
  }
}

1
An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2
An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3
The default gateway to route egress network traffic to.
4
An array describing routes to configure inside the pod.
5
The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6
The gateway where network traffic is routed.
7
Optional: DNS configuration.
8
An of array of one or more IP addresses for to send DNS queries to.
9
The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10
An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.
10.5.1.2.2. Dynamic IP address assignment configuration

The following JSON describes the configuration for dynamic IP address address assignment with DHCP.

Renewal of DHCP leases

A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.

To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:

Example shim network attachment definition

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  ...
  additionalNetworks:
  - name: dhcp-shim
    namespace: default
    type: Raw
    rawCNIConfig: |-
      {
        "name": "dhcp-shim",
        "cniVersion": "0.3.1",
        "type": "bridge",
        "ipam": {
          "type": "dhcp"
        }
      }

DHCP assignment configuration

{
  "ipam": {
    "type": "dhcp"
  }
}

10.5.1.2.3. Dynamic IP address assignment configuration with Whereabouts

The Whereabouts CNI plug-in allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.

The following JSON describes the configuration for dynamic IP address assignment with Whereabouts:

Whereabouts assignment configuration

{
  "ipam": {
    "type": "whereabouts",
    "range": "<range>", 1
    "exclude": ["<exclude_part>, ..."], 2
  }
}

1
Specify an IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
2
Optional: Specify a list of IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
10.5.1.2.4. Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.7"
        }
      ]
  }
}
10.5.1.2.5. Dynamic IP address assignment configuration example using DHCP

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "dhcp"
  }
}
10.5.1.2.6. Dynamic IP address assignment configuration example using Whereabouts

You can configure ipam to use Whereabouts:

{
  "ipam": {
    "type": "whereabouts",
    "range": "192.0.2.192/27",
    "exclude": [
       "192.0.2.192/30",
       "192.0.2.196/32"
    ]
  }
}

10.5.2. Next steps

10.6. Configuring an ipvlan network

As a cluster administrator, you can configure an additional network for your cluster by using the ipvlan Container Network Interface (CNI) plug-in. The virtual network created by this plug-in is associated with a physical interface that you specify.

10.6.1. Creating an additional network attachment with the ipvlan CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition object automatically.

Important

Do not edit the NetworkAttachmentDefinition objects that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the ipvlan CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: 1
      - name: test-network-1
        namespace: test-1
        type: Raw
        rawCNIConfig: '{
          "cniVersion": "0.3.1",
          "name": "test-network-1",
          "type": "ipvlan",
          "master": "eth1",
          "mode": "l2",
          "ipam": {
            "type": "static",
            "addresses": [
              {
                "address": "192.168.1.23/24"
              }
            ]
          }
        }'
    1
    Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.
  4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

    $ oc get network-attachment-definitions -n <namespace>

    Example output

    NAME                 AGE
    test-network-1       14m

10.6.1.1. Configuration for ipvlan

The configuration for an additional network attachment that uses the ipvlan Container Network Interface (CNI) plug-in is provided in two parts:

  • Cluster Network Operator (CNO) configuration
  • CNI plug-in configuration

The CNO configuration specifies the name for the additional network attachment and the namespace to create the attachment in. The plug-in is configured by a JSON object specified by the rawCNIConfig parameter in the CNO configuration.

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration

name: <name> 1
namespace: <namespace> 2
rawCNIConfig: '{ 3
  ...
}'
type: Raw

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If you do not specify a value, then the default namespace is used.
3
Specify the CNI plug-in configuration in JSON format, which is based on the following template.

The following object describes the configuration parameters for the ipvlan CNI plug-in:

ipvlan CNI plug-in JSON configuration object

{
  "cniVersion": "0.3.1",
  "name": "<name>", 1
  "type": "ipvlan",
  "mode": "<mode>", 2
  "master": "<master>", 3
  "mtu": <mtu>, 4
  "ipam": { 5
    ...
  }
}

1
Specify the value for the name parameter you provided previously for the CNO configuration.
2
Specify the operating mode for the virtual network. The value must be l2, l3, or l3s. The default value is l2.
3
Specify the ethernet interface to associate with the network attachment. If a master is not specified, the interface for the default network route is used.
4
Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
5
Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition.
10.6.1.1.1. ipvlan configuration example

The following example configures an additional network named ipvlan-net:

name: ipvlan-net
namespace: work-network
type: Raw
rawCNIConfig: '{ 1
  "cniVersion": "0.3.1",
  "name": "work-network",
  "type": "ipvlan",
  "master": "eth1",
  "mode": "l3",
  "ipam": {
    "type": "dhcp"
    }
}'
1
The CNI configuration object is specified as a YAML string.

10.6.1.2. Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

You can use the following methods for IP address assignment:

  • Static assignment.
  • Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.
  • Dynamic assignment through the Whereabouts IPAM CNI plug-in.
10.6.1.2.1. Static IP address assignment configuration

The following JSON describes the configuration for static IP address assignment:

Static assignment configuration

{
  "ipam": {
    "type": "static",
    "addresses": [ 1
      {
        "address": "<address>", 2
        "gateway": "<gateway>" 3
      }
    ],
    "routes": [ 4
      {
        "dst": "<dst>", 5
        "gw": "<gw>" 6
      }
    ],
    "dns": { 7
      "nameservers": ["<nameserver>"], 8
      "domain": "<domain>", 9
      "search": ["<search_domain>"] 10
    }
  }
}

1
An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2
An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3
The default gateway to route egress network traffic to.
4
An array describing routes to configure inside the pod.
5
The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6
The gateway where network traffic is routed.
7
Optional: DNS configuration.
8
An of array of one or more IP addresses for to send DNS queries to.
9
The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10
An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.
10.6.1.2.2. Dynamic IP address assignment configuration

The following JSON describes the configuration for dynamic IP address address assignment with DHCP.

Renewal of DHCP leases

A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.

To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:

Example shim network attachment definition

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  ...
  additionalNetworks:
  - name: dhcp-shim
    namespace: default
    type: Raw
    rawCNIConfig: |-
      {
        "name": "dhcp-shim",
        "cniVersion": "0.3.1",
        "type": "bridge",
        "ipam": {
          "type": "dhcp"
        }
      }

DHCP assignment configuration

{
  "ipam": {
    "type": "dhcp"
  }
}

10.6.1.2.3. Dynamic IP address assignment configuration with Whereabouts

The Whereabouts CNI plug-in allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.

The following JSON describes the configuration for dynamic IP address assignment with Whereabouts:

Whereabouts assignment configuration

{
  "ipam": {
    "type": "whereabouts",
    "range": "<range>", 1
    "exclude": ["<exclude_part>, ..."], 2
  }
}

1
Specify an IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
2
Optional: Specify a list of IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
10.6.1.2.4. Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.7"
        }
      ]
  }
}
10.6.1.2.5. Dynamic IP address assignment configuration example using DHCP

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "dhcp"
  }
}
10.6.1.2.6. Dynamic IP address assignment configuration example using Whereabouts

You can configure ipam to use Whereabouts:

{
  "ipam": {
    "type": "whereabouts",
    "range": "192.0.2.192/27",
    "exclude": [
       "192.0.2.192/30",
       "192.0.2.196/32"
    ]
  }
}

10.6.2. Next steps

10.7. Configuring a macvlan network with basic customizations

As a cluster administrator, you can configure an additional network for your cluster using the macvlan Container Network Interface (CNI) plug-in. When a pod is attached to the network, the plug-in creates a sub-interface from the parent interface on the host. A unique hardware mac address is generated for each sub-device.

Important

The unique MAC addresses this plug-in generates for sub-interfaces might not be compatible with the security polices of your cloud provider.

You specify a basic configuration directly in YAML. This approach offers fewer configuration options than by specifying a macvlan configuration by using a CNI object directly in JSON.

10.7.1. Creating an additional network attachment with the macvlan CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition object automatically.

Important

Do not edit the NetworkAttachmentDefinition objects that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the macvlan CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: 1
      - name: test-network-1
        namespace: test-1
        type: SimpleMacvlan
        simpleMacvlanConfig:
          ipamConfig:
            type: static
            staticIPAMConfig:
              addresses:
              - address: 10.1.1.7/24
    1
    Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.
  4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

    $ oc get network-attachment-definitions -n <namespace>

    Example output

    NAME                 AGE
    test-network-1       14m

10.7.1.1. Configuration for macvlan CNI plug-in

The following YAML describes the configuration parameters for the macvlan Container Network Interface (CNI) plug-in:

macvlan YAML configuration

name: <name> 1
namespace: <namespace> 2
type: SimpleMacvlan
simpleMacvlanConfig:
  master: <master> 3
  mode: <mode> 4
  mtu: <mtu> 5
  ipamConfig: 6
    ...

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If a value is not specified, the default namespace is used.
3
The ethernet interface to associate with the virtual interface. If a value for master is not specified, then the host system’s primary ethernet interface is used.
4
Configures traffic visibility on the virtual network. Must be either bridge, passthru, private, or vepa. If a value for mode is not provided, the default value is bridge.
5
Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
6
Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition.
10.7.1.1.1. macvlan configuration example

The following example configures an additional network named macvlan-net:

name: macvlan-net
namespace: work-network
type: SimpleMacvlan
simpleMacvlanConfig:
  ipamConfig:
    type: DHCP

10.7.1.2. Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

The following YAML configuration describes the parameters that you can set.

ipam CNI plug-in YAML configuration object

ipamConfig:
  type: <type> 1
  ... 2

1
Specify static to configure the plug-in to manage IP address assignment. Specify DHCP to allow a DHCP server to manage IP address assignment. You cannot specify any additional parameters if you specify a value of DHCP.
2
If you set the type parameter to static, then provide the staticIPAMConfig parameter.
10.7.1.2.1. Static ipam configuration YAML

The following YAML describes a configuration for static IP address assignment:

Static ipam configuration YAML

ipamConfig:
  type: static
  staticIPAMConfig:
    addresses: 1
    - address: <address> 2
      gateway: <gateway> 3
    routes: 4
    - destination: <destination> 5
      gateway: <gateway> 6
    dns: 7
      nameservers: 8
      - <nameserver>
      domain: <domain> 9
      search: 10
      - <search_domain>

1
A collection of mappings that define IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2
An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3
The default gateway to route egress network traffic to.
4
A collection of mappings describing routes to configure inside the pod.
5
The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6
The gateway where network traffic is routed.
7
Optional: The DNS configuration.
8
A collection of one or more IP addresses for to send DNS queries to.
9
The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10
An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.
10.7.1.2.2. Dynamic ipam configuration YAML

The following YAML describes a configuration for static IP address assignment:

Dynamic ipam configuration YAML

ipamConfig:
  type: DHCP

10.7.1.2.3. Static IP address assignment configuration example

The following example shows an ipam configuration for static IP addresses:

ipamConfig:
  type: static
  staticIPAMConfig:
    addresses:
    - address: 198.51.100.11/24
      gateway: 198.51.100.10
    routes:
    - destination: 0.0.0.0/0
      gateway: 198.51.100.1
    dns:
      nameservers:
      - 198.51.100.1
      - 198.51.100.2
      domain: testDNS.example
      search:
      - testdomain1.example
      - testdomain2.example
10.7.1.2.4. Dynamic IP address assignment configuration example

The following example shows an ipam configuration for DHCP:

ipamConfig:
  type: DHCP

10.7.2. Next steps

10.8. Configuring a macvlan network

As a cluster administrator, you can configure an additional network for your cluster using the macvlan Container Network Interface (CNI) plug-in with advanced customization. When a pod is attached to the network, the plug-in creates a sub-interface from the parent interface on the host. A unique hardware mac address is generated for each sub-device.

Important

The unique MAC addresses this plug-in generates for sub-interfaces might not be compatible with the security polices of your cloud provider.

You specify a configuration with a CNI object. This approach allows you to specify additional configuration options that are not available when using a YAML configuration.

10.8.1. Creating an additional network attachment with the macvlan CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition object automatically.

Important

Do not edit the NetworkAttachmentDefinition objects that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the macvlan CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: 1
      - name: test-network-1
        namespace: test-1
        type: Raw
        rawCNIConfig: '{
          "cniVersion": "0.3.1",
          "name": "test-network-1",
          "type": "macvlan",
          "master": "eth1",
          "ipam": {
            "type": "static",
            "addresses": [
              {
                "address": "192.168.1.23/24"
              }
            ]
          }
        }'
    1
    Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.
  4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

    $ oc get network-attachment-definitions -n <namespace>

    Example output

    NAME                 AGE
    test-network-1       14m

10.8.1.1. Configuration for macvlan CNI plug-in

The configuration for an additional network attachment that uses the macvlan Container Network Interface (CNI) plug-in is provided in two parts:

  • Cluster Network Operator (CNO) configuration
  • CNI plug-in configuration

The CNO configuration specifies the name for the additional network attachment and the namespace to create the attachment in. The plug-in is configured by a JSON object specified by the rawCNIConfig parameter in the CNO configuration.

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration

name: <name> 1
namespace: <namespace> 2
rawCNIConfig: '{ 3
  ...
}'
type: Raw

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Specify the namespace to create the network attachment in. If you do not specify a value, then the default namespace is used.
3
Specify the CNI plug-in configuration in JSON format, which is based on the following template.

The following object describes the configuration parameters for the macvlan CNI plug-in:

macvlan CNI plug-in JSON configuration object

{
  "cniVersion": "0.3.1",
  "name": "<name>", 1
  "type": "macvlan",
  "mode": "<mode>", 2
  "master": "<master>", 3
  "mtu": <mtu>, 4
  "ipam": { 5
    ...
  }
}

1
Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2
Configures traffic visibility on the virtual network. Must be either bridge, passthru, private, or vepa. If a value is not provided, the default value is bridge.
3
The ethernet interface to associate with the virtual interface. If a value is not specified, then the host system’s primary ethernet interface is used.
4
Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
5
Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the attachment definition.
10.8.1.1.1. macvlan configuration example

The following example configures an additional network named macvlan-net:

name: macvlan-net
namespace: work-network
type: Raw
rawCNIConfig: |-
  {
    "cniVersion": "0.3.1",
    "name": "macvlan-net",
    "type": "macvlan",
    "master": "eth1",
    "mode": "bridge",
    "ipam": {
      "type": "dhcp"
      }
  }

10.8.1.2. Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

You can use the following methods for IP address assignment:

  • Static assignment.
  • Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.
  • Dynamic assignment through the Whereabouts IPAM CNI plug-in.
10.8.1.2.1. Static IP address assignment configuration

The following JSON describes the configuration for static IP address assignment:

Static assignment configuration

{
  "ipam": {
    "type": "static",
    "addresses": [ 1
      {
        "address": "<address>", 2
        "gateway": "<gateway>" 3
      }
    ],
    "routes": [ 4
      {
        "dst": "<dst>", 5
        "gw": "<gw>" 6
      }
    ],
    "dns": { 7
      "nameservers": ["<nameserver>"], 8
      "domain": "<domain>", 9
      "search": ["<search_domain>"] 10
    }
  }
}

1
An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2
An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3
The default gateway to route egress network traffic to.
4
An array describing routes to configure inside the pod.
5
The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6
The gateway where network traffic is routed.
7
Optional: DNS configuration.
8
An of array of one or more IP addresses for to send DNS queries to.
9
The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10
An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.
10.8.1.2.2. Dynamic IP address assignment configuration

The following JSON describes the configuration for dynamic IP address address assignment with DHCP.

Renewal of DHCP leases

A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.

To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:

Example shim network attachment definition

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  ...
  additionalNetworks:
  - name: dhcp-shim
    namespace: default
    type: Raw
    rawCNIConfig: |-
      {
        "name": "dhcp-shim",
        "cniVersion": "0.3.1",
        "type": "bridge",
        "ipam": {
          "type": "dhcp"
        }
      }

DHCP assignment configuration

{
  "ipam": {
    "type": "dhcp"
  }
}

10.8.1.2.3. Dynamic IP address assignment configuration with Whereabouts

The Whereabouts CNI plug-in allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.

The following JSON describes the configuration for dynamic IP address assignment with Whereabouts:

Whereabouts assignment configuration

{
  "ipam": {
    "type": "whereabouts",
    "range": "<range>", 1
    "exclude": ["<exclude_part>, ..."], 2
  }
}

1
Specify an IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
2
Optional: Specify a list of IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.
10.8.1.2.4. Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.7"
        }
      ]
  }
}
10.8.1.2.5. Dynamic IP address assignment configuration example using DHCP

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "dhcp"
  }
}
10.8.1.2.6. Dynamic IP address assignment configuration example using Whereabouts

You can configure ipam to use Whereabouts:

{
  "ipam": {
    "type": "whereabouts",
    "range": "192.0.2.192/27",
    "exclude": [
       "192.0.2.192/30",
       "192.0.2.196/32"
    ]
  }
}

10.8.2. Next steps

10.9. Editing an additional network

As a cluster administrator you can modify the configuration for an existing additional network.

10.9.1. Modifying an additional network attachment definition

As a cluster administrator, you can make changes to an existing additional network. Any existing pods attached to the additional network will not be updated.

Prerequisites

  • You have configured an additional network for your cluster.
  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To edit an additional network for your cluster, complete the following steps:

  1. Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:

    $ oc edit networks.operator.openshift.io cluster
  2. In the additionalNetworks collection, update the additional network with your changes.
  3. Save your changes and quit the text editor to commit your changes.
  4. Optional: Confirm that the CNO updated the NetworkAttachmentDefinition object by running the following command. Replace <network-name> with the name of the additional network to display. There might be a delay before the CNO updates the NetworkAttachmentDefinition object to reflect your changes.

    $ oc get network-attachment-definitions <network-name> -o yaml

    For example, the following console output displays a NetworkAttachmentDefinition object that is named net1:

    $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
    { "cniVersion": "0.3.1", "type": "macvlan",
    "master": "ens5",
    "mode": "bridge",
    "ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }

10.10. Removing an additional network

As a cluster administrator you can remove an additional network attachment.

10.10.1. Removing an additional network attachment definition

As a cluster administrator, you can remove an additional network from your OpenShift Container Platform cluster. The additional network is not removed from any pods it is attached to.

Prerequisites

  • Install the OpenShift CLI (oc).
  • Log in as a user with cluster-admin privileges.

Procedure

To remove an additional network from your cluster, complete the following steps:

  1. Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR by removing the configuration from the additionalNetworks collection for the network attachment definition you are removing.

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: [] 1
    1
    If you are removing the configuration mapping for the only additional network attachment definition in the additionalNetworks collection, you must specify an empty collection.
  3. Save your changes and quit the text editor to commit your changes.
  4. Optional: Confirm that the additional network CR was deleted by running the following command:

    $ oc get network-attachment-definition --all-namespaces