How to configure and test LACP bonding with Open vSwitch DPDK?

Solution In Progress - Updated -

Environment

Red Hat Enterprise Linux 7.4
Open vSwitch 2.6.1

Issue

How to configure and test LACP bonding with Open vSwitch DPDK?

Resolution

Note: This knowledge base article demonstrates switch configuration with a Dell S4048-ON switch. Whereas configuration of RHEL and OVS will remain the same, different switch vendors' operating systems will use a different syntax to configure LACP.

Note: OVS bonds with LACP may not be supported depending on the version of Red Hat OpenStack Platform (RHOSP). More details about LACP supportability in RHOSP can be found in this technical article Supportability of LACP (802.3ad) in Red Hat OpenStack Platform.

Configuring the switch ports for LACP

Configuring the switch ports for LACP. First, reset switch interfaces to defaults:

S4048-ON-sw#config t                                                                                                                                                                                       
S4048-ON-sw(conf)#default int te1/2
S4048-ON-sw(conf)#default int te1/7

Configure port-channel and some other port settings:

S4048-ON-sw(conf)#int range te1/2,te1/7
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)#port-channel-protocol lacp
S4048-ON-sw(conf-if-range-te-1/2,te-1/7-lacp)#
S4048-ON-sw(conf-if-range-te-1/2,te-1/7-lacp)#port-channel 1 mode active
S4048-ON-sw(conf-if-range-te-1/2,te-1/7-lacp)#end
S4048-ON-sw#config t
S4048-ON-sw(conf)#int range te1/2,te1/7
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)# no ip address
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)# mtu 9216
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)# flowcontrol rx on tx off
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)# no shutdown
S4048-ON-sw(conf-if-range-te-1/2,te-1/7)#end
S4048-ON-sw#show run int te1/2
!
interface TenGigabitEthernet 1/2
 no ip address
 mtu 9216
 flowcontrol rx on tx off
!  
 port-channel-protocol LACP
  port-channel 1 mode active
 no shutdown

Configure VLANs:

S4048-ON-sw#config t
S4048-ON-sw(conf)#int range vlan901-909         
S4048-ON-sw(conf-if-range-vl-901-909)#tagged Port-channel 1
S4048-ON-sw(conf-if-range-vl-901-909)#end
S4048-ON-sw#

Verify VLAN tagging:

S4048-ON-sw#show vlan id 902

Codes: * - Default VLAN, G - GVRP VLANs, R - Remote Port Mirroring VLANs, P - Primary, C - Community, I - Isolated
       O - Openflow, Vx - Vxlan
Q: U - Untagged, T - Tagged
   x - Dot1x untagged, X - Dot1x tagged
   o - OpenFlow untagged, O - OpenFlow tagged
   G - GVRP tagged, M - Vlan-stack
   i - Internal untagged, I - Internal tagged, v - VLT untagged, V - VLT tagged

    NUM    Status    Description                     Q Ports
    902    Active    Tenant                          T Po1()
                                                     T Te 1/1,1/3-1/6,1/8-1/20

Verify LACP configuration:

S4048-ON-sw#show lacp 1
Port-channel 1 admin up, oper down, mode lacp
LACP Fast Switch-Over Disabled
Actor   System ID:  Priority 32768, Address 1418.7789.9a8a
Partner System ID:  Priority 0, Address 0000.0000.0000
Actor Admin Key 1, Oper Key 1, Partner Oper Key 1, VLT Peer Oper Key 1
LACP LAG 1 is an individual link

LACP LAG 1 is a normal LAG

A - Active LACP, B - Passive LACP, C - Short Timeout, D - Long Timeout
E - Aggregatable Link, F - Individual Link, G - IN_SYNC, H - OUT_OF_SYNC
I - Collection enabled, J - Collection disabled, K - Distribution enabled
L - Distribution disabled, M - Partner Defaulted, N - Partner Non-defaulted,
O - Receiver is in expired state, P - Receiver is not in expired state

Port Te 1/2 is disabled, LACP is disabled and mode is lacp
Port State: Not in Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEHJLMP Key 1 Priority 32768
  Partner is not present

Port Te 1/7 is enabled, LACP is enabled and mode is lacp
Port State: Not in Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEHJLMP Key 1 Priority 32768
  Partner is not present

Configuring Linux kernel bonding for LACP as a baseline

As a preliminary step, it is always good to start with a simpler scenario. It is easier to configure Linux kernel bonding a baseline and verify the switch and RHEL can form an LACP bond.

Move all interfaces to kernel space and test with kernel space bonding. In this example, p1p1 maps to bus address 0000:04:00.0 and p1p2 maps to bus address 0000:04:00.1.

[root@baremetal ~]# driverctl unset-override 0000:04:00.0
[root@baremetal ~]# driverctl unset-override 0000:04:00.1

Load the bonding driver, configure a bond interface bond10 and enslave interfaces p1p1 and p1p2:

[root@baremetal ~]# modprobe bonding miimon=100 mode=4 lacp_rate=1                                                                                                                                     
[root@baremetal ~]# ip link add name bond10 type bond                                                                                                                                                  
[root@baremetal ~]# ifenslave bond10 p1p1 p1p2                                                                                                                                                          
Illegal operation; the specified master interface 'bond10' is not up.
[root@baremetal ~]# ip link set dev bond10 up
[root@baremetal ~]# ifenslave bond10 p1p1 p1p2 

Verify LACP from RHEL:

[root@baremetal ~]# cat /proc/net/bonding/bond10
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: a0:36:9f:e3:dd:c8
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 2
        Actor Key: 13
        Partner Key: 1
        Partner Mac Address: 14:18:77:89:9a:8a

Slave Interface: p1p1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:e3:dd:c8
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: a0:36:9f:e3:dd:c8
    port key: 13
    port priority: 255
    port number: 1
    port state: 63
details partner lacp pdu:
    system priority: 32768
    system mac address: 14:18:77:89:9a:8a
    oper key: 1
    port priority: 32768
    port number: 203
    port state: 63

Slave Interface: p1p2
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a0:36:9f:e3:dd:ca
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: monitoring
Partner Churn State: monitoring
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: a0:36:9f:e3:dd:c8
    port key: 13
    port priority: 255
    port number: 2
    port state: 63
details partner lacp pdu:
    system priority: 32768
    system mac address: 14:18:77:89:9a:8a
    oper key: 1
    port priority: 32768
    port number: 208
    port state: 63

Verify LACP from the switch:

S4048-ON-sw#show lacp 1          
Port-channel 1 admin up, oper up, mode lacp
LACP Fast Switch-Over Disabled
Actor   System ID:  Priority 32768, Address 1418.7789.9a8a
Partner System ID:  Priority 65535, Address a036.9fe3.ddc8
Actor Admin Key 1, Oper Key 1, Partner Oper Key 13, VLT Peer Oper Key 1
LACP LAG 1 is an aggregatable link
LACP LAG 1 is a normal LAG

A - Active LACP, B - Passive LACP, C - Short Timeout, D - Long Timeout
E - Aggregatable Link, F - Individual Link, G - IN_SYNC, H - OUT_OF_SYNC
I - Collection enabled, J - Collection disabled, K - Distribution enabled
L - Distribution disabled, M - Partner Defaulted, N - Partner Non-defaulted,
O - Receiver is in expired state, P - Receiver is not in expired state

Port Te 1/2 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ACEGIKNP Key 13 Priority 255

Port Te 1/7 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ACEGIKNP Key 13 Priority 255
S4048-ON-sw#

Remove the bonding configuration:

[root@baremetal ~]# ip link del dev bond10
[root@baremetal ~]#

Note: One can change the bonding mode by following: https://access.redhat.com/solutions/28529

Configuring OVS DPDK bonding for LACP

The next step is to configure an LACP bond within OVS DPDK.

Prepare Open vSwitch

Make sure that huge pages etc. are configured in RHEL:

[root@baremetal bonding]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.17.1.el7.x86_64 root=UUID=fa414390-f78d-49d4-a164-54615a32977b ro console=tty0 console=ttyS0,115200n8 crashkernel=auto rhgb quiet default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 skew_tick=1 nohz=on nohz_full=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 rcu_nocbs=2,4,6,8,10,12,14,16,18,22,24,26,28,30,32,34,36,38,3,5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 tuned.non_isolcpus=00300003 intel_pstate=disable nosoftlockup

Configure OVS for DPDK:

[root@baremetal bonding]# ovs-vsctl list Open_vSwitch | grep other
other_config        : {}
[root@baremetal bonding]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init="true"
[root@baremetal bonding]# ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x17c0017c
[root@baremetal bonding]# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0x00000001

Switch interfaces into user space:

[root@baremetal bonding]# ethtool -i p1p1 | grep bus
bus-info: 0000:04:00.0
[root@baremetal bonding]# ethtool -i p1p2 | grep bus                                                                                                
bus-info: 0000:04:00.1
[root@baremetal bonding]# driverctl set-override 0000:04:00.0 vfio-pci                                                                              
[root@baremetal bonding]# driverctl set-override 0000:04:00.1 vfio-pci

Restart Open vSwitch, journalctl -u ovs-vswitchd -f & running in the background:

[root@baremetal bonding]# systemctl restart openvswitch
Apr 19 13:02:49 baremetal systemd[1]: Stopping Open vSwitch Forwarding Unit...
Apr 19 13:02:49 baremetal systemd[1]: Stopping Open vSwitch Forwarding Unit...
Apr 19 13:02:49 baremetal ovs-ctl[91399]: Exiting ovs-vswitchd (91202) [  OK  ]
Apr 19 13:02:49 baremetal ovs-ctl[91399]: Exiting ovs-vswitchd (91202) [  OK  ]
Apr 19 13:02:49 baremetal systemd[1]: Starting Open vSwitch Forwarding Unit...
Apr 19 13:02:49 baremetal systemd[1]: Starting Open vSwitch Forwarding Unit...
Apr 19 13:02:49 baremetal ovs-ctl[91483]: Starting ovs-vswitchd EAL: Detected 40 lcore(s)
Apr 19 13:02:49 baremetal ovs-ctl[91483]: Starting ovs-vswitchd EAL: Detected 40 lcore(s)
Apr 19 13:02:49 baremetal ovs-ctl[91483]: EAL: Probing VFIO support...
Apr 19 13:02:49 baremetal ovs-vswitchd[91509]: EAL: Probing VFIO support...
Apr 19 13:02:49 baremetal ovs-ctl[91483]: EAL: VFIO support initialized
Apr 19 13:02:49 baremetal ovs-vswitchd[91509]: EAL: VFIO support initialized
Apr 19 13:02:49 baremetal ovs-ctl[91483]: EAL: Probing VFIO support...
Apr 19 13:02:49 baremetal ovs-vswitchd[91509]: EAL: Probing VFIO support...
Apr 19 13:02:49 baremetal ovs-ctl[91483]: EAL: VFIO support initialized
Apr 19 13:02:49 baremetal ovs-vswitchd[91509]: EAL: VFIO support initialized
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: PCI device 0000:04:00.0 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: PCI device 0000:04:00.0 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:04:00.0 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   using IOMMU type 1 (Type 1)
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   using IOMMU type 1 (Type 1)
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:04:00.0 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   using IOMMU type 1 (Type 1)
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   using IOMMU type 1 (Type 1)
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: Ignore mapping IO port bar(2) addr: 3021
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: Ignore mapping IO port bar(2) addr: 3021
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: Ignore mapping IO port bar(2) addr: 3021
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: Ignore mapping IO port bar(2) addr: 3021
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:04:00.1 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:04:00.1 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: PCI device 0000:04:00.1 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: PCI device 0000:04:00.1 on NUMA socket 0
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: Ignore mapping IO port bar(2) addr: 3001
Apr 19 13:02:59 baremetal ovs-ctl[91483]: EAL: Ignore mapping IO port bar(2) addr: 3001
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: Ignore mapping IO port bar(2) addr: 3001
Apr 19 13:02:59 baremetal ovs-vswitchd[91509]: EAL: Ignore mapping IO port bar(2) addr: 3001
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL: PCI device 0000:05:00.0 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL: PCI device 0000:05:00.0 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:05:00.0 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:05:00.0 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL: PCI device 0000:05:00.1 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL: PCI device 0000:05:00.1 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-ctl[91483]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:05:00.1 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL: PCI device 0000:05:00.1 on NUMA socket 0
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-vswitchd[91509]: EAL:   probe driver: 8086:154d net_ixgbe
Apr 19 13:03:00 baremetal ovs-ctl[91483]: [  OK  ]
Apr 19 13:03:00 baremetal ovs-ctl[91483]: [  OK  ]
Apr 19 13:03:00 baremetal ovs-ctl[91483]: Enabling remote OVSDB managers [  OK  ]
Apr 19 13:03:00 baremetal ovs-ctl[91483]: Enabling remote OVSDB managers [  OK  ]
Apr 19 13:03:00 baremetal systemd[1]: Started Open vSwitch Forwarding Unit.
Apr 19 13:03:00 baremetal systemd[1]: Started Open vSwitch Forwarding Unit.
[root@baremetal bonding]#

Configure LACP bond

Add bond:

[root@baremetal bonding]# ovs-vsctl add-br ovsbr0  -- set bridge ovsbr0  datapath_type=netdev
[root@baremetal bonding]# ovs-vsctl add-bond ovsbr0 dpdkbond dpdk0 dpdk1     bond_mode=balance-tcp     lacp=active -- set interface dpdk0 type=dpdk     -- set Interface dpdk1 type=dpdk

Verify from Open vSwitch:

[root@baremetal bonding]# ovs-appctl lacp/show dpdkbond                                                                                                                              
---- dpdkbond ----
        status: active negotiated
        sys_id: a0:36:9f:e3:dd:c8
        sys_priority: 65534
        aggregation key: 1
        lacp_time: slow

slave: dpdk0: current attached
        port_id: 2
        port_priority: 65535
        may_enable: true

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 2
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation synchronized collecting distributing

        partner sys_id: 14:18:77:89:9a:8a
        partner sys_priority: 32768
        partner port_id: 203
        partner port_priority: 32768
        partner key: 1
        partner state: activity timeout aggregation synchronized collecting distributing

slave: dpdk1: current attached
        port_id: 1
        port_priority: 65535
        may_enable: true

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 1
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation synchronized collecting distributing

        partner sys_id: 14:18:77:89:9a:8a
        partner sys_priority: 32768
        partner port_id: 208
        partner port_priority: 32768
        partner key: 1
        partner state: activity timeout aggregation synchronized collecting distributing

[root@baremetal bonding]# ovs-appctl bond/show dpdkbond                                                                                                                                                                                                                                                                                                                                                            
---- dpdkbond ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 6817 ms
lacp_status: negotiated
active slave mac: a0:36:9f:e3:dd:c8(dpdk0)

slave dpdk0: enabled
        active slave
        may_enable: true

slave dpdk1: enabled
        may_enable: true

Verify from the switch:

S4048-ON-sw#show lacp 1     
Port-channel 1 admin up, oper up, mode lacp
LACP Fast Switch-Over Disabled
Actor   System ID:  Priority 32768, Address 1418.7789.9a8a
Partner System ID:  Priority 65534, Address a036.9fe3.ddc8
Actor Admin Key 1, Oper Key 1, Partner Oper Key 1, VLT Peer Oper Key 1
LACP LAG 1 is an aggregatable link
LACP LAG 1 is a normal LAG

A - Active LACP, B - Passive LACP, C - Short Timeout, D - Long Timeout
E - Aggregatable Link, F - Individual Link, G - IN_SYNC, H - OUT_OF_SYNC
I - Collection enabled, J - Collection disabled, K - Distribution enabled
L - Distribution disabled, M - Partner Defaulted, N - Partner Non-defaulted,
O - Receiver is in expired state, P - Receiver is not in expired state

Port Te 1/2 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ADEGIKNP Key 1 Priority 65535

Port Te 1/7 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ADEGIKNP Key 1 Priority 65535
S4048-ON-sw#

Enabling / disabling ports from OVS

Individual ports can be enables or shut down with ovs-ofctl mod-port <bridge> <port> [up|down]

Shut down a port:

[root@baremetal bonding]#  ovs-ofctl mod-port ovsbr0 dpdk1 down

Verify:

[root@baremetal bonding]# ovs-appctl lacp/show dpdkbond                                                                                                            
---- dpdkbond ----
        status: active negotiated
        sys_id: a0:36:9f:e3:dd:c8
        sys_priority: 65534
        aggregation key: 1
        lacp_time: slow

slave: dpdk0: current attached
        port_id: 2
        port_priority: 65535
        may_enable: true

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 2
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation synchronized collecting distributing

        partner sys_id: 14:18:77:89:9a:8a
        partner sys_priority: 32768
        partner port_id: 203
        partner port_priority: 32768
        partner key: 1
        partner state: activity timeout aggregation synchronized collecting distributing

slave: dpdk1: defaulted detached
        port_id: 1
        port_priority: 65535
        may_enable: false

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 1
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation defaulted

        partner sys_id: 00:00:00:00:00:00
        partner sys_priority: 0
        partner port_id: 0
        partner port_priority: 0
        partner key: 0
        partner state:
[root@baremetal bonding]# ovs-appctl bond/show dpdkbond
---- dpdkbond ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 3315 ms
lacp_status: negotiated
active slave mac: a0:36:9f:e3:dd:c8(dpdk0)

slave dpdk0: enabled
        active slave
        may_enable: true

slave dpdk1: disabled
        may_enable: false

Verify on the switch:

S4048-ON-sw#show lacp 1
Port-channel 1 admin up, oper up, mode lacp
LACP Fast Switch-Over Disabled
Actor   System ID:  Priority 32768, Address 1418.7789.9a8a
Partner System ID:  Priority 65534, Address a036.9fe3.ddc8
Actor Admin Key 1, Oper Key 1, Partner Oper Key 1, VLT Peer Oper Key 1
LACP LAG 1 is an aggregatable link
LACP LAG 1 is a normal LAG

A - Active LACP, B - Passive LACP, C - Short Timeout, D - Long Timeout
E - Aggregatable Link, F - Individual Link, G - IN_SYNC, H - OUT_OF_SYNC
I - Collection enabled, J - Collection disabled, K - Distribution enabled
L - Distribution disabled, M - Partner Defaulted, N - Partner Non-defaulted,
O - Receiver is in expired state, P - Receiver is not in expired state

Port Te 1/2 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ADEGIKNP Key 1 Priority 65535

Port Te 1/7 is disabled, LACP is disabled and mode is lacp
Port State: Not in Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEHJLNP Key 1 Priority 32768
  Partner is not present

Bring up the port again:

[root@baremetal bonding]#  ovs-ofctl mod-port ovsbr0 dpdk1 up

Verify from RHEL:

[root@baremetal bonding]# ovs-appctl bond/show dpdkbond                                                                                                            
---- dpdkbond ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
updelay: 0 ms
downdelay: 0 ms
next rebalance: 7846 ms
lacp_status: negotiated
active slave mac: a0:36:9f:e3:dd:c8(dpdk0)

slave dpdk0: enabled
        active slave
        may_enable: true

slave dpdk1: enabled
        may_enable: true

[root@baremetal bonding]# ovs-appctl lacp/show dpdkbond                                                                                                            
---- dpdkbond ----
        status: active negotiated
        sys_id: a0:36:9f:e3:dd:c8
        sys_priority: 65534
        aggregation key: 1
        lacp_time: slow

slave: dpdk0: current attached
        port_id: 2
        port_priority: 65535
        may_enable: true

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 2
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation synchronized collecting distributing

        partner sys_id: 14:18:77:89:9a:8a
        partner sys_priority: 32768
        partner port_id: 203
        partner port_priority: 32768
        partner key: 1
        partner state: activity timeout aggregation synchronized collecting distributing

slave: dpdk1: current attached
        port_id: 1
        port_priority: 65535
        may_enable: true

        actor sys_id: a0:36:9f:e3:dd:c8
        actor sys_priority: 65534
        actor port_id: 1
        actor port_priority: 65535
        actor key: 1
        actor state: activity aggregation synchronized collecting distributing

        partner sys_id: 14:18:77:89:9a:8a
        partner sys_priority: 32768
        partner port_id: 208
        partner port_priority: 32768
        partner key: 1
        partner state: activity timeout aggregation synchronized collecting distributing

Verify from the switch:

S4048-ON-sw#show lacp 1
Port-channel 1 admin up, oper up, mode lacp
LACP Fast Switch-Over Disabled
Actor   System ID:  Priority 32768, Address 1418.7789.9a8a
Partner System ID:  Priority 65534, Address a036.9fe3.ddc8
Actor Admin Key 1, Oper Key 1, Partner Oper Key 1, VLT Peer Oper Key 1
LACP LAG 1 is an aggregatable link
LACP LAG 1 is a normal LAG

A - Active LACP, B - Passive LACP, C - Short Timeout, D - Long Timeout
E - Aggregatable Link, F - Individual Link, G - IN_SYNC, H - OUT_OF_SYNC
I - Collection enabled, J - Collection disabled, K - Distribution enabled
L - Distribution disabled, M - Partner Defaulted, N - Partner Non-defaulted,
O - Receiver is in expired state, P - Receiver is not in expired state

Port Te 1/2 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ADEGIKNP Key 1 Priority 65535

Port Te 1/7 is enabled, LACP is enabled and mode is lacp
Port State: Bundle
  Actor   Admin: State ACEHJLMP Key 1 Priority 32768
           Oper: State ACEGIKNP Key 1 Priority 32768
  Partner Admin: State BDFHJLMP Key 0 Priority 0
           Oper: State ADEGIKNP Key 1 Priority 65535

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments