Appendix B. Using Flannel

As an alternate to the default SDN, Red Hat OpenShift Container Platform provides Ansible playbooks for installing flannel-based networking. This is useful when running Red Hat OpenShift Container Platform within a cloud provider platform that relies on SDN, such as Red Hat OpenShift Container Platform, and want to avoid encapsulating packets twice through both platforms.

Note

The following section are from OpenShift Documentation - Using Flannel site. For the latest version of this section, reference the link directly.

Flannel uses a single IP network space for all of the containers allocating a contiguous subset of the space to each instance. Consequently, nothing prevents a container from attempting to contact any IP address in the same network space. This hinders multi-tenancy because the network cannot be used to isolate containers in one application from another.

Depending on preference, isolation or performance, determine the appropriate choice when deciding between the different OpenShift SDN plugins and flannel options for internal networks.

Important

The current version of neutron enforces port security on ports by default. This prevents the port from sending or receiving packets with a MAC address different from that on the port itself. Flannel creates virtual MACs and IP addresses and must send and receive packets on the port, thus port security must be disabled on the ports that carry flannel traffic.

To enable flannel within an Red Hat OpenShift Container Platform cluster:

  1. neutron port security controls must be configured to be compatible with Flannel. The default configuration of Red Hat OpenShift Container Platform disables user control of port_security. Configure netruon to allow users to control the port_security setting on individual ports.

    1. On the neutron servers, add the following to the /etc/neutron/plugins/ml2/ml2_conf.ini file: 

      [ml2]
...
extension_drivers = port_security

    2. Then, restart the neutron services: 

      service neutron-dhcp-agent restart
service neutron-ovs-cleanup restart
service neutron-metadata-agentrestart
service neutron-l3-agent restart
service neutron-plugin-openvswitch-agent restart
service neutron-vpn-agent restart
service neutron-server  restart

  2. When creating the Red Hat OpenShift Container Platform instances on Red Hat OpenStack Platform, disable both port security and security groups in the ports where the container network Flannel interface resides: 

    neutron port-update $port --no-security-groups --port-security-enabled=False
    Note

    Flannel gather information from etcd to configure and assign the subnets in the nodes. Therefore, the security group attached to the etcd hosts should allow access from nodes to port 2379/tcp, and nodes security group should allow egress communication to that port on the etcd hosts.

    1. Set the following variables in the Ansible inventory file before running the installation: 

      openshift_use_openshift_sdn=false 1
openshift_use_flannel=true 2
flannel_interface=eth0
      1
      Set openshift_use_openshift_sdn to false to disable the default SDN.
      2
      Set openshift_use_flannel to true to enable flannel in place.

    2. Optionally, specify the interface to use for inter-host communication using the flannel_interface variable. Without this variable, the Red Hat OpenShift Container Platform installation uses the default interface.

      Note

      Custom networking CIDR for pods and services using flannel are to be supported in a future release. BZ#1473858

  3. After the Red Hat OpenShift Container Platform installation, add a set of iptables rules on every Red Hat OpenShift Container Platform node: 

    iptables -A DOCKER -p all -j ACCEPT
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

    To persist those changes in the /etc/sysconfig/iptables use the following command on every node: 

    cp /etc/sysconfig/iptables{,.orig}
sh -c "tac /etc/sysconfig/iptables.orig | sed -e '0,/:DOCKER -/ s/:DOCKER -/:DOCKER ACCEPT/' | awk '"\!"p && /POSTROUTING/{print \"-A POSTROUTING -o eth1 -j MASQUERADE\"; p=1} 1' | tac > /etc/sysconfig/iptables"
    Note

    The iptables-save command saves all the current in memory iptables rules. However, because Docker, Kubernetes and Red Hat OpenShift Container Platform create a high number of iptables rules (services, etc.) not designed to be persisted, saving these rules can become problematic.

To isolate container traffic from the rest of the Red Hat OpenShift Container Platform traffic, Red Hat recommends creating an isolated tenant network and attaching all the nodes to it. If using a different network interface (eth1), ensure to configure the interface to start at boot time through the /etc/sysconfig/network-scripts/ifcfg-eth1 file: 

DEVICE=eth1
TYPE=Ethernet
BOOTPROTO=dhcp
ONBOOT=yes
DEFTROUTE=no
PEERDNS=no