Chapter 21. Configuring Fibre Channel over Ethernet

Based on the IEEE T11 FC-BB-5 standard, Fibre Channel over Ethernet (FCoE) is a protocol to transmit Fibre Channel frames over Ethernet networks. Typically, data centers have a dedicated LAN and Storage Area Network (SAN) that are separated from each other with their own specific configuration. FCoE combines these networks into a single and converged network structure. Benefits of FCoE are, for example, lower hardware and energy costs.

21.1. Using hardware FCoE HBAs in RHEL

In Red Hat Enterprise Linux you can use hardware FCoE Host Bus Adapter (HBA) supported by the following drivers:

  • qedf
  • bnx2fc
  • fnic

If you use such a HBA, you configure the FCoE settings in the setup of the HBA. For details, see the documentation of the adapter.

After you configured the HBA in its setup, the exported Logical Unit Numbers (LUN) from the Storage Area Network (SAN) are automatically available to RHEL as /dev/sd* devices. You can use these devices similar to local storage devices.

21.2. Setting up a software FCoE device

A software FCoE device enables you to access Logical Unit Numbers (LUN) over FCoE using an Ethernet adapter that partially supports FCoE offload.

Important

RHEL does not support software FCoE devices that require the fcoe.ko kernel module. For details, see FCoE software removal in the Considerations in adopting RHEL 8 documentation.

After you complete this procedure, the exported LUNs from the Storage Area Network (SAN) are automatically available to RHEL as /dev/sd* devices. You can use these devices similar to local storage devices.

Prerequisites

  • The Host Bus Adapter (HBA) uses the qedf, bnx2fc, or fnic driver and does not require the fcoe.ko kernel module.
  • The SAN uses a VLAN to separate the storage traffic from normal Ethernet traffic.
  • The network switch has been configured to support the VLAN.
  • The HBA of the server is configured in its BIOS. For details, see the documentation of your HBA.
  • The HBA is connected to the network and the link is up.

Procedure

  1. Install the fcoe-utils package:

    # yum install fcoe-utils
  2. Copy the /etc/fcoe/cfg-ethx template file to /etc/fcoe/cfg-interface_name. For example, if you want to configure the enp1s0 interface to use FCoE, enter:

    # cp /etc/fcoe/cfg-ethx /etc/fcoe/cfg-enp1s0
  3. Enable and start the fcoe service:

    # systemctl enable --now fcoe
  4. Discover the FCoE VLAN ID, start the initiator, and create a network device for the discovered VLAN:

    # fipvlan -s -c enp1s0
    Created VLAN device enp1s0.200
    Starting FCoE on interface enp1s0.200
    Fibre Channel Forwarders Discovered
    interface       | VLAN | FCF MAC
    ------------------------------------------
    enp1s0          | 200  | 00:53:00:a7:e7:1b
  5. Optional: To display details about the discovered targets, the LUNs, and the devices associated with the LUNs, enter:

    # fcoeadm -t
    Interface:        enp1s0.200
    Roles:            FCP Target
    Node Name:        0x500a0980824acd15
    Port Name:        0x500a0982824acd15
    Target ID:        0
    MaxFrameSize:     2048 bytes
    OS Device Name:   rport-11:0-1
    FC-ID (Port ID):  0xba00a0
    State:            Online
    
    LUN ID  Device Name   Capacity   Block Size  Description
    ------  -----------  ----------  ----------  ---------------------
         0  sdb           28.38 GiB      512     NETAPP LUN (rev 820a)
         ...

    This example shows that LUN 0 from the SAN has been attached to the host as the /dev/sdb device.

Verification steps

  • Use the fcoeadm -i command to display information about all active FCoE interfaces:

    # fcoeadm -i
    Description:      BCM57840 NetXtreme II 10 Gigabit Ethernet
    Revision:         11
    Manufacturer:     Broadcom Inc. and subsidiaries
    Serial Number:    000AG703A9B7
    
    Driver:           bnx2x Unknown
    Number of Ports:  1
    
        Symbolic Name:     bnx2fc (QLogic BCM57840) v2.12.13 over enp1s0.200
        OS Device Name:    host11
        Node Name:         0x2000000af70ae935
        Port Name:         0x2001000af70ae935
        Fabric Name:       0x20c8002a6aa7e701
        Speed:             10 Gbit
        Supported Speed:   1 Gbit, 10 Gbit
        MaxFrameSize:      2048 bytes
        FC-ID (Port ID):   0xba02c0
        State:             Online

Additional resources

  • For further details about the fcoeadm utility, see the fcoeadm(8) man page.
  • For details about how to mount storage connected through a software FCoE when the system boots, see the /usr/share/doc/fcoe-utils/README file.

21.3. Additional resources