How to configure OVN-Kubernetes internal IP address subnets at install time or post-installation

Solution Verified - Updated -

Environment

  • Red Hat OpenShift Container Platform (RHOCP)
    • 4.14
  • OVN-Kubernetes

Issue

  • For the CNI OVN-Kubernetes, the default internal subnets are:

    • internalJoinSubnet (IPv4): 100.64.0.0/16
    • internalJoinSubnet (IPv6): fd98::/64
    • internalTransitSwitchSubnet (IPv4): 100.88.0.0/16
    • internalTransitSwitchSubnet (IPv6): fd97::/64
  • These subnets may conflict with existing infrastructure networks. This article covers how to change them either at install time or post-installation.

Resolution

Option 1: Configure at install time using install-config.yaml (OCP 4.17 and later)

  • From OCP 4.17 onwards, both internalJoinSubnet and internalTransitSwitchSubnet can be configured directly in install-config.yaml under networking.ovnKubernetesConfig:
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 192.168.0.0/24
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
  ovnKubernetesConfig:
    ipv4:
      internalJoinSubnet: 10.104.240.0/22
      internalTransitSwitchSubnet: 10.104.244.0/22
  • To explore all available fields:
./openshift-install explain installconfig.networking.ovnKubernetesConfig.ipv4
  • Note: In OCP 4.19 and above, the installer validates that clusterNetwork, serviceNetwork, and machineNetwork do not overlap with the OVN internal subnets, and will interrupt installation if an overlap is detected. Plan your subnet ranges accordingly before installation.

Option 2: Configure at install time using a custom manifest (OCP 4.14 and later)

  • This method is required when using the Assisted Installer (which does not expose ovnKubernetesConfig in its native UI), or when using an older OCP version that does not support the install-config.yaml stanza.

  • Create the install-config and manifest files:

INST_DIR="</path/to/installation/directory>"
mkdir $INST_DIR
./openshift-install create install-config --dir $INST_DIR
./openshift-install create manifests --dir $INST_DIR
  • Create a new manifest file from the existing cluster network config:
cd $INST_DIR
cp manifests/cluster-network-02-config.yml manifests/cluster-network-03-config.yml
sed -i 's%config.openshift.io/v1%operator.openshift.io/v1%g' manifests/cluster-network-03-config.yml
  • Edit cluster-network-03-config.yml to add the desired subnets under spec.defaultNetwork:
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  creationTimestamp: null
  name: cluster
spec:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  externalIP:
    policy: {}
  networkType: OVNKubernetes
  serviceNetwork:
  - 172.30.0.0/16
  defaultNetwork:
    type: OVNKubernetes
    ovnKubernetesConfig:
      ipv4:
        internalJoinSubnet: 10.104.240.0/22
        internalTransitSwitchSubnet: 10.104.244.0/22
status: {}
  • Note: For OCP 4.14 and 4.15, use the legacy field name v4InternalSubnet instead of ipv4.internalJoinSubnet. The internalTransitSwitchSubnet field was not available for install-time configuration prior to OCP 4.17.

  • For Assisted Installer deployments, upload this manifest via the Assisted Installer UI custom manifests step or via the API endpoint /v2/clusters/{cluster_id}/manifests.

  • Run the installation:

./openshift-install create cluster --dir $INST_DIR --log-level=info
  • For Assisted Installer deployments via the API (no UI), base64-encode the manifest and upload it to the cluster before triggering installation using the /v2/clusters/{cluster_id}/manifests endpoint:
MANIFEST_CONTENT=$(base64 -w0 cluster-network-ovn-subnets.yaml)

curl -s -X POST \
  "https://api.openshift.com/api/assisted-install/v2/clusters/${CLUSTER_ID}/manifests" \
  -H "Authorization: Bearer ${API_TOKEN}" \
  -H "Content-Type: application/json" \
  -d "{
    \"file_name\": \"cluster-network-ovn-subnets.yaml\",
    \"folder\": \"manifests\",
    \"content\": \"${MANIFEST_CONTENT}\"
  }"
  • Verify the manifest was uploaded before proceeding with the installation:
curl -s \
  "https://api.openshift.com/api/assisted-install/v2/clusters/${CLUSTER_ID}/manifests" \
  -H "Authorization: Bearer ${API_TOKEN}" | jq .

For the complete Assisted Installer API workflow, refer to the official documentation: Installing with the Assisted Installer API


Option 3: Post-installation configuration (Day 2)

  • Both subnets can also be reconfigured on a running cluster using oc patch. This operation can take up to 30 minutes to complete.
oc patch network.operator.openshift.io cluster --type='merge' \
  -p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalJoinSubnet": "10.104.240.0/22"}}}}}'

oc patch network.operator.openshift.io cluster --type='merge' \
  -p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalTransitSwitchSubnet": "10.104.244.0/22"}}}}}'
  • Both can be patched in a single command:
oc patch network.operator.openshift.io cluster --type='merge' \
  -p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalJoinSubnet": "10.104.240.0/22","internalTransitSwitchSubnet": "10.104.244.0/22"}}}}}'

Root Cause

  • The default OVN-Kubernetes internal subnets (100.64.0.0/16 for internalJoinSubnet and 100.88.0.0/16 for internalTransitSwitchSubnet) conflict with existing infrastructure or corporate network ranges, requiring custom values to be set at install time or post-installation.

Diagnostic Steps

  • To review the current OVN-Kubernetes internal subnet configuration on a running cluster:
oc get network.operator.openshift.io -o jsonpath="{.items[0].spec.defaultNetwork}" | jq .
  • Example output showing both fields configured:
{
  "ovnKubernetesConfig": {
    "egressIPConfig": {},
    "gatewayConfig": {
      "routingViaHost": false
    },
    "genevePort": 6081,
    "mtu": 1400,
    "policyAuditConfig": {
      "destination": "null",
      "maxFileSize": 50,
      "maxLogFiles": 5,
      "rateLimit": 20,
      "syslogFacility": "local0"
    },
    "ipv4": {
      "internalJoinSubnet": "10.104.240.0/22",
      "internalTransitSwitchSubnet": "10.104.244.0/22"
    }
  },
  "type": "OVNKubernetes"
}

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments