How to create VLAN interface for VMs in OpenShift Virtualization?

Resolution

• There are two ways to configure VLAN/s for VMs.

1. With VLAN Bridge filtering.
2. Without VLAN Bridge filtering.

1] With VLAN Bridge filtering.

With the VLAN filtering feature, we only need one bridge interface and no VLAN interfaces. The bridge will handle the VLAN filtering.

• Create the bridge interface over physical nic, enable VLAN filter, and attach the interface to the bridge directly.

       ---------
       | ens4f0 |
       ---------
          | 
       ------
       | br1 | 
       ------
          |
          |
    ------------------
    |                |
 ---------          ---------
| VM1 NIC |        | VM2 NIC | 
 ---------          ---------

• Use a NodeNetworkConfigurationPolicy manifest YAML file to create the Linux bridge. By default, VLAN filtering will be enabled on the bridge.

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: br1-ens4f0-policy 
spec:
  desiredState:
    interfaces:
      - name: br1 
        description: Linux bridge with ens4f0 as a port 
        type: linux-bridge 
        state: up 
        ipv4:
          enabled: false 
        bridge:
          options:
            stp:
              enabled: false 
          port:
            - name: ens4f0 

• Create the NetworkAttachmentDefinition with the required VLAN for the VM and replace <bridge-interface> with "br1" interface defined in the nncp:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-1223 
  annotations:
    k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/<bridge-interface> 
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "net-1223",  
    "type": "cnv-bridge", 
    "bridge": "<bridge-interface>", 
    "macspoofchk": true, 
    "vlan": 1223          <<<<
  }'

• Multiple Network Attachment Definition can be created over the same bridge for different VLANs.

2] Without VLAN Bridge filtering.

• This is a traditional method where we create VLAN interfaces separately and the bridge is created on top of VLAN interfaces.

       ---------
       | ens4f0 |
       ---------
          | 
       --------------
       | ens4f0.1223 |
       --------------
          |
       ------
       | br1 |
       ------
          |
          |
    ------------------
    |                |
---------           ---------
|VM1 NIC |          | VM2 NIC | 
---------           ---------

• In the nncp, we first have to create a VLAN interface and then we have to create a bridge on top of this VLAN interface.

• Example YAML for bridge br1:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: example-nncp
spec:
  desiredState:
    interfaces:
      - name: ens4f0.1223    <<< VLAN interface
        state: up
        type: vlan
        vlan:
          base-iface: ens4f0
          id: 1223
      - bridge:
          options:
            stp:
              enabled: false
          port:
          - name: ens4f0.1223
            vlan: {}         <<< Disabling bridge VLAN
        description: Linux bridge with ens4f0 as a port on VLAN 1223
        name: br1
        state: up
        type: linux-bridge

• Create additional VLAN interfaces and bridge interfaces for additional VLANs.

The NetworkAttachmentDefinition will be created without the VLAN parameter.

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-1223 
  annotations:
    k8s.v1.cni.cncf.io/resourceName: bridge.network.kubevirt.io/<bridge-interface> 
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "net-1223", 
    "type": "cnv-bridge", 
    "bridge": "<bridge-interface>", 
    "macspoofchk": true 
  }'

Refer Connecting a VM to a Linux bridge network

For understanding the difference between these 2 types of VLAN mechanisms, please read additional information on:
VLAN filter support on the bridge