How to configure OVN-Kubernetes internal IP address subnets at install time or post-installation
Environment
- Red Hat OpenShift Container Platform (RHOCP)
- 4.14
- OVN-Kubernetes
Issue
-
For the CNI OVN-Kubernetes, the default internal subnets are:
internalJoinSubnet(IPv4):100.64.0.0/16internalJoinSubnet(IPv6):fd98::/64internalTransitSwitchSubnet(IPv4):100.88.0.0/16internalTransitSwitchSubnet(IPv6):fd97::/64
-
These subnets may conflict with existing infrastructure networks. This article covers how to change them either at install time or post-installation.
Resolution
Option 1: Configure at install time using install-config.yaml (OCP 4.17 and later)
- From OCP 4.17 onwards, both
internalJoinSubnetandinternalTransitSwitchSubnetcan be configured directly ininstall-config.yamlundernetworking.ovnKubernetesConfig:
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 192.168.0.0/24
networkType: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16
ovnKubernetesConfig:
ipv4:
internalJoinSubnet: 10.104.240.0/22
internalTransitSwitchSubnet: 10.104.244.0/22
- To explore all available fields:
./openshift-install explain installconfig.networking.ovnKubernetesConfig.ipv4
- Note: In OCP 4.19 and above, the installer validates that
clusterNetwork,serviceNetwork, andmachineNetworkdo not overlap with the OVN internal subnets, and will interrupt installation if an overlap is detected. Plan your subnet ranges accordingly before installation.
Option 2: Configure at install time using a custom manifest (OCP 4.14 and later)
-
This method is required when using the Assisted Installer (which does not expose
ovnKubernetesConfigin its native UI), or when using an older OCP version that does not support theinstall-config.yamlstanza. -
Create the install-config and manifest files:
INST_DIR="</path/to/installation/directory>"
mkdir $INST_DIR
./openshift-install create install-config --dir $INST_DIR
./openshift-install create manifests --dir $INST_DIR
- Create a new manifest file from the existing cluster network config:
cd $INST_DIR
cp manifests/cluster-network-02-config.yml manifests/cluster-network-03-config.yml
sed -i 's%config.openshift.io/v1%operator.openshift.io/v1%g' manifests/cluster-network-03-config.yml
- Edit
cluster-network-03-config.ymlto add the desired subnets underspec.defaultNetwork:
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
creationTimestamp: null
name: cluster
spec:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
externalIP:
policy: {}
networkType: OVNKubernetes
serviceNetwork:
- 172.30.0.0/16
defaultNetwork:
type: OVNKubernetes
ovnKubernetesConfig:
ipv4:
internalJoinSubnet: 10.104.240.0/22
internalTransitSwitchSubnet: 10.104.244.0/22
status: {}
-
Note: For OCP 4.14 and 4.15, use the legacy field name
v4InternalSubnetinstead ofipv4.internalJoinSubnet. TheinternalTransitSwitchSubnetfield was not available for install-time configuration prior to OCP 4.17. -
For Assisted Installer deployments, upload this manifest via the Assisted Installer UI custom manifests step or via the API endpoint
/v2/clusters/{cluster_id}/manifests. -
Run the installation:
./openshift-install create cluster --dir $INST_DIR --log-level=info
- For Assisted Installer deployments via the API (no UI), base64-encode the manifest and upload it to the cluster before triggering installation using the
/v2/clusters/{cluster_id}/manifestsendpoint:
MANIFEST_CONTENT=$(base64 -w0 cluster-network-ovn-subnets.yaml)
curl -s -X POST \
"https://api.openshift.com/api/assisted-install/v2/clusters/${CLUSTER_ID}/manifests" \
-H "Authorization: Bearer ${API_TOKEN}" \
-H "Content-Type: application/json" \
-d "{
\"file_name\": \"cluster-network-ovn-subnets.yaml\",
\"folder\": \"manifests\",
\"content\": \"${MANIFEST_CONTENT}\"
}"
- Verify the manifest was uploaded before proceeding with the installation:
curl -s \
"https://api.openshift.com/api/assisted-install/v2/clusters/${CLUSTER_ID}/manifests" \
-H "Authorization: Bearer ${API_TOKEN}" | jq .
For the complete Assisted Installer API workflow, refer to the official documentation: Installing with the Assisted Installer API
Option 3: Post-installation configuration (Day 2)
- Both subnets can also be reconfigured on a running cluster using
oc patch. This operation can take up to 30 minutes to complete.
oc patch network.operator.openshift.io cluster --type='merge' \
-p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalJoinSubnet": "10.104.240.0/22"}}}}}'
oc patch network.operator.openshift.io cluster --type='merge' \
-p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalTransitSwitchSubnet": "10.104.244.0/22"}}}}}'
- Both can be patched in a single command:
oc patch network.operator.openshift.io cluster --type='merge' \
-p='{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"ipv4":{"internalJoinSubnet": "10.104.240.0/22","internalTransitSwitchSubnet": "10.104.244.0/22"}}}}}'
Root Cause
- The default OVN-Kubernetes internal subnets (
100.64.0.0/16forinternalJoinSubnetand100.88.0.0/16forinternalTransitSwitchSubnet) conflict with existing infrastructure or corporate network ranges, requiring custom values to be set at install time or post-installation.
Diagnostic Steps
- To review the current OVN-Kubernetes internal subnet configuration on a running cluster:
oc get network.operator.openshift.io -o jsonpath="{.items[0].spec.defaultNetwork}" | jq .
- Example output showing both fields configured:
{
"ovnKubernetesConfig": {
"egressIPConfig": {},
"gatewayConfig": {
"routingViaHost": false
},
"genevePort": 6081,
"mtu": 1400,
"policyAuditConfig": {
"destination": "null",
"maxFileSize": 50,
"maxLogFiles": 5,
"rateLimit": 20,
"syslogFacility": "local0"
},
"ipv4": {
"internalJoinSubnet": "10.104.240.0/22",
"internalTransitSwitchSubnet": "10.104.244.0/22"
}
},
"type": "OVNKubernetes"
}
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments