Chapter 5. Monitoring OVN

You can use the ovn-trace command to monitor and troubleshoot OVN logical flows, and you can use the ovs-ofctl dump-flows command to monitor and troubleshoot OpenFlows.

5.1. Creating aliases for OVN troubleshooting commands

OVN database commands (such as ovn-nbctl show) run on the ovn_controller container. The container runs on the controller node and compute nodes. To simplify your access to the commands, create and source a script that defines aliases.

Prerequisites

  • New deployment of Red Hat OpenStack Platform 16.0 or higher, with ML2/OVN as the default mechanism driver.

Creating and using OVN database command aliases

  1. Create a shell script file in the appropriate directory on the overcloud node where you want to run the ovn commands. For example, log in to the controller node as heat-admin and create the file ovn-alias.sh in the heat-admin user’s ~/bin directory.
  2. Save the following commands in the script file.

    EXTERNAL_ID=\
    $(sudo ovs-vsctl get open . external_ids:ovn-remote | awk -F: '{print $2}')
    export NBDB=tcp:${EXTERNAL_ID}:6641
    export SBDB=tcp:${EXTERNAL_ID}:6642
    
    alias ovn-sbctl="sudo podman exec ovn_controller ovn-sbctl --db=$SBDB"
    alias ovn-nbctl="sudo podman exec ovn_controller ovn-nbctl --db=$NBDB"
    alias ovn-trace="sudo podman exec ovn_controller ovn-trace --db=$SBDB"
  3. Source the script file. For example, log in to the controller node as heat-admin and run the following command.

    # source ovn-alias.sh
  4. Validate an alias. For example, show the northbound database.

    ovn-nbctl show

    Example output

    switch 26ce22db-1795-41bd-b561-9827cbd81778 (neutron-f8e79863-6c58-43d0-8f7d-8ec4a423e13b) (aka internal_network)
    	port 1913c3ae-8475-4b60-a479-df7bcce8d9c8
        	addresses: ["fa:16:3e:33:c1:fc 192.168.254.76"]
    	port 1aabaee3-b944-4da2-bf0a-573215d3f3d9
        	addresses: ["fa:16:3e:16:cb:ce 192.168.254.74"]
    	port 7e000980-59f9-4a0f-b76a-4fdf4e86f27b
        	type: localport
        	addresses: ["fa:16:3e:c9:30:ed 192.168.254.2"]

5.2. Monitoring OVN logical flows

OVN uses logical flows that are tables of flows with a priority, match, and actions. These logical flows are distributed to the ovn-controller running on each Compute node. You can use the ovn-sbctl lflow-list command on the Controller node to view the full set of logical flows.

Prerequisites

  • RHOSP deployment with ML2/OVN.

Procedure

  1. On the Controller node, run the command ovn-sbctl --db=tcp:172.17.1.10:6642 lflow-list.
  2. Inspect the output.

    $ ovn-sbctl --db=tcp:172.17.1.10:6642 lflow-list
        Datapath: "sw0" (d7bf4a7b-e915-4502-8f9d-5995d33f5d10)  Pipeline: ingress
          table=0 (ls_in_port_sec_l2  ), priority=100  , match=(eth.src[40]), action=(drop;)
          table=0 (ls_in_port_sec_l2  ), priority=100  , match=(vlan.present), action=(drop;)
          table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport == "sw0-port1" && eth.src == {00:00:00:00:00:01}), action=(next;)
          table=0 (ls_in_port_sec_l2  ), priority=50   , match=(inport == "sw0-port2" && eth.src == {00:00:00:00:00:02}), action=(next;)
          table=1 (ls_in_port_sec_ip  ), priority=0    , match=(1), action=(next;)
          table=2 (ls_in_port_sec_nd  ), priority=90   , match=(inport == "sw0-port1" && eth.src == 00:00:00:00:00:01 && arp.sha == 00:00:00:00:00:01), action=(next;)
          table=2 (ls_in_port_sec_nd  ), priority=90   , match=(inport == "sw0-port1" && eth.src == 00:00:00:00:00:01 && ip6 && nd && ((nd.sll == 00:00:00:00:00:00 || nd.sll == 00:00:00:00:00:01) || ((nd.tll == 00:00:00:00:00:00 || nd.tll == 00:00:00:00:00:01)))), action=(next;)
          table=2 (ls_in_port_sec_nd  ), priority=90   , match=(inport == "sw0-port2" && eth.src == 00:00:00:00:00:02 && arp.sha == 00:00:00:00:00:02), action=(next;)
          table=2 (ls_in_port_sec_nd  ), priority=90   , match=(inport == "sw0-port2" && eth.src == 00:00:00:00:00:02 && ip6 && nd && ((nd.sll == 00:00:00:00:00:00 || nd.sll == 00:00:00:00:00:02) || ((nd.tll == 00:00:00:00:00:00 || nd.tll == 00:00:00:00:00:02)))), action=(next;)
          table=2 (ls_in_port_sec_nd  ), priority=80   , match=(inport == "sw0-port1" && (arp || nd)), action=(drop;)
          table=2 (ls_in_port_sec_nd  ), priority=80   , match=(inport == "sw0-port2" && (arp || nd)), action=(drop;)
          table=2 (ls_in_port_sec_nd  ), priority=0    , match=(1), action=(next;)
          table=3 (ls_in_pre_acl      ), priority=0    , match=(1), action=(next;)
          table=4 (ls_in_pre_lb       ), priority=0    , match=(1), action=(next;)
          table=5 (ls_in_pre_stateful ), priority=100  , match=(reg0[0] == 1), action=(ct_next;)
          table=5 (ls_in_pre_stateful ), priority=0    , match=(1), action=(next;)
          table=6 (ls_in_acl          ), priority=0    , match=(1), action=(next;)
          table=7 (ls_in_qos_mark     ), priority=0    , match=(1), action=(next;)
          table=8 (ls_in_lb           ), priority=0    , match=(1), action=(next;)
          table=9 (ls_in_stateful     ), priority=100  , match=(reg0[1] == 1), action=(ct_commit(ct_label=0/1); next;)
          table=9 (ls_in_stateful     ), priority=100  , match=(reg0[2] == 1), action=(ct_lb;)
          table=9 (ls_in_stateful     ), priority=0    , match=(1), action=(next;)
          table=10(ls_in_arp_rsp      ), priority=0    , match=(1), action=(next;)
          table=11(ls_in_dhcp_options ), priority=0    , match=(1), action=(next;)
          table=12(ls_in_dhcp_response), priority=0    , match=(1), action=(next;)
          table=13(ls_in_l2_lkup      ), priority=100  , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
          table=13(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:00:01), action=(outport = "sw0-port1"; output;)
          table=13(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:00:02), action=(outport = "sw0-port2"; output;)
        Datapath: "sw0" (d7bf4a7b-e915-4502-8f9d-5995d33f5d10)  Pipeline: egress
          table=0 (ls_out_pre_lb      ), priority=0    , match=(1), action=(next;)
          table=1 (ls_out_pre_acl     ), priority=0    , match=(1), action=(next;)
          table=2 (ls_out_pre_stateful), priority=100  , match=(reg0[0] == 1), action=(ct_next;)
          table=2 (ls_out_pre_stateful), priority=0    , match=(1), action=(next;)
          table=3 (ls_out_lb          ), priority=0    , match=(1), action=(next;)
          table=4 (ls_out_acl         ), priority=0    , match=(1), action=(next;)
          table=5 (ls_out_qos_mark    ), priority=0    , match=(1), action=(next;)
          table=6 (ls_out_stateful    ), priority=100  , match=(reg0[1] == 1), action=(ct_commit(ct_label=0/1); next;)
          table=6 (ls_out_stateful    ), priority=100  , match=(reg0[2] == 1), action=(ct_lb;)
          table=6 (ls_out_stateful    ), priority=0    , match=(1), action=(next;)
          table=7 (ls_out_port_sec_ip ), priority=0    , match=(1), action=(next;)
          table=8 (ls_out_port_sec_l2 ), priority=100  , match=(eth.mcast), action=(output;)
          table=8 (ls_out_port_sec_l2 ), priority=50   , match=(outport == "sw0-port1" && eth.dst == {00:00:00:00:00:01}), action=(output;)
          table=8 (ls_out_port_sec_l2 ), priority=50   , match=(outport == "sw0-port2" && eth.dst == {00:00:00:00:00:02}), action=(output;)

Key differences between OVN and OpenFlow include:

  • OVN ports are logical entities that reside somewhere on a network, not physical ports on a single switch.
  • OVN gives each table in the pipeline a name in addition to its number. The name describes the purpose of that stage in the pipeline.
  • The OVN match syntax supports complex Boolean expressions.
  • The actions supported in OVN logical flows extend beyond those of OpenFlow. You can implement higher level features, such as DHCP, in the OVN logical flow syntax.

ovn-trace

The ovn-trace command can simulate how a packet travels through the OVN logical flows, or help you determine why a packet is dropped. Provide the ovn-trace command with the following parameters:

DATAPATH
The logical switch or logical router where the simulated packet starts.
MICROFLOW
The simulated packet, in the syntax used by the ovn-sb database.

This example displays the --minimal output option on a simulated packet and shows that the packet reaches its destination:

$ ovn-trace --minimal sw0 'inport == "sw0-port1" && eth.src == 00:00:00:00:00:01 && eth.dst == 00:00:00:00:00:02'
    # reg14=0x1,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,dl_type=0x0000
    output("sw0-port2");

In more detail, the --summary output for this same simulated packet shows the full execution pipeline:

$ ovn-trace --summary sw0 'inport == "sw0-port1" && eth.src == 00:00:00:00:00:01 && eth.dst == 00:00:00:00:00:02'
# reg14=0x1,vlan_tci=0x0000,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,dl_type=0x0000
ingress(dp="sw0", inport="sw0-port1") {
    outport = "sw0-port2";
    output;
    egress(dp="sw0", inport="sw0-port1", outport="sw0-port2") {
        output;
        /* output to "sw0-port2", type "" */;
    };
};

The example output shows:

  • The packet enters the sw0 network from the sw0-port1 port and runs the ingress pipeline.
  • The outport variable is set to sw0-port2 indicating that the intended destination for this packet is sw0-port2.
  • The packet is output from the ingress pipeline, which brings it to the egress pipeline for sw0 with the outport variable set to sw0-port2.
  • The output action is executed in the egress pipeline, which outputs the packet to the current value of the outport variable, which is sw0-port2.

Additional resources

  • See the ovn-trace man page for complete details.

5.3. Monitoring OpenFlows

You can use ovs-ofctl dump-flows command to monitor the OpenFlow flows on a logical switch in your network.

Prerequisites

  • RHOSP deployment with ML2/OVN.

Procedure

  1. On the Controller node, run the command ovs-ofctl dump-flows br-int.
  2. Inspect the output, which will resemble the following example.

    $ ovs-ofctl dump-flows br-int
    NXST_FLOW reply (xid=0x4):
     cookie=0x0, duration=72.132s, table=0, n_packets=0, n_bytes=0, idle_age=72, priority=10,in_port=1,dl_src=00:00:00:00:00:01 actions=resubmit(,1)
     cookie=0x0, duration=60.565s, table=0, n_packets=0, n_bytes=0, idle_age=60, priority=10,in_port=2,dl_src=00:00:00:00:00:02 actions=resubmit(,1)
     cookie=0x0, duration=28.127s, table=0, n_packets=0, n_bytes=0, idle_age=28, priority=0 actions=drop
     cookie=0x0, duration=13.887s, table=1, n_packets=0, n_bytes=0, idle_age=13, priority=0,in_port=1 actions=output:2
     cookie=0x0, duration=4.023s, table=1, n_packets=0, n_bytes=0, idle_age=4, priority=0,in_port=2 actions=output:1