Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

20.3. Adding a Network Device

Network device driver modules are loaded automatically by udev.
You can add a network interface on IBM Z dynamically or persistently.
  • Dynamically
    1. Load the device driver
    2. Remove the network devices from the list of ignored devices.
    3. Create the group device.
    4. Configure the device.
    5. Set the device online.
  • Persistently
    1. Create a configuration script.
    2. Activate the interface.
The following sections provide basic information for each task of each IBM Z network device driver. Section 20.3.1, “Adding a qeth Device” describes how to add a qeth device to an existing instance of Red Hat Enterprise Linux. Section 20.3.2, “Adding an LCS Device” describes how to add an lcs device to an existing instance of Red Hat Enterprise Linux.

20.3.1. Adding a qeth Device

The qeth network device driver supports IBM Z OSA-Express features in QDIO mode, HiperSockets, z/VM guest LAN, and z/VM VSWITCH.
The qeth device driver assigns the same interface name for Ethernet and Hipersockets devices: enccwbus_ID. The bus ID is composed of the channel subsystem ID, subchannel set ID, and device number, for example enccw0.0.0a00.

20.3.1.1. Dynamically Adding a qeth Device

To add a qeth device dynamically, follow these steps:
  1. Determine whether the qeth device driver modules are loaded. The following example shows loaded qeth modules: 
    # lsmod | grep qeth
											qeth_l3                  127056  9
											qeth_l2                   73008  3
											ipv6                  492872  155ip6t_REJECT,nf_conntrack_ipv6,qeth_l3
											qeth                  115808  2 qeth_l3,qeth_l2
											qdio                   68240  1 qeth
											ccwgroup               12112  2 qeth
    If the output of the lsmod command shows that the qeth modules are not loaded, run the modprobe command to load them: 
    # modprobe qeth
  2. Use the cio_ignore utility to remove the network channels from the list of ignored devices and make them visible to Linux: 
    # cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
    Replace read_device_bus_id,write_device_bus_id,data_device_bus_id with the three device bus IDs representing a network device. For example, if the read_device_bus_id is 0.0.f500, the write_device_bus_id is 0.0.f501, and the data_device_bus_id is 0.0.f502: 
    # cio_ignore -r 0.0.f500,0.0.f501,0.0.f502
  3. Use the znetconf utility to sense and list candidate configurations for network devices: 
    # znetconf -u
Scanning for network devices...
Device IDs                 Type    Card Type      CHPID Drv.
------------------------------------------------------------
0.0.f500,0.0.f501,0.0.f502 1731/01 OSA (QDIO)        00 qeth
0.0.f503,0.0.f504,0.0.f505 1731/01 OSA (QDIO)        01 qeth
0.0.0400,0.0.0401,0.0.0402 1731/05 HiperSockets      02 qeth
  4. Select the configuration you want to work with and use znetconf to apply the configuration and to bring the configured group device online as network device. 
    # znetconf -a f500
Scanning for network devices...
Successfully configured device 0.0.f500 (enccw0.0.f500)
  5. Optionally, you can also pass arguments that are configured on the group device before it is set online: 
    # znetconf -a f500 -o portname=myname
Scanning for network devices...
Successfully configured device 0.0.f500 (enccw0.0.f500)
    Now you can continue to configure the enccw0.0.f500 network interface.
Alternatively, you can use sysfs attributes to set the device online as follows:
  1. Create a qeth group device: 
    # echo read_device_bus_id,write_device_bus_id,data_device_bus_id > /sys/bus/ccwgroup/drivers/qeth/group
    For example: 
    # echo 0.0.f500,0.0.f501,0.0.f502 > /sys/bus/ccwgroup/drivers/qeth/group
  2. Next, verify that the qeth group device was created properly by looking for the read channel: 
    # ls /sys/bus/ccwgroup/drivers/qeth/0.0.f500
    You can optionally set additional parameters and features, depending on the way you are setting up your system and the features you require, such as:
    • portno
    • layer2
    • portname
  3. Bring the device online by writing 1 to the online sysfs attribute: 
    # echo 1 > /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online
  4. Then verify the state of the device: 
    # cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/online
											1
    A return value of 1 indicates that the device is online, while a return value 0 indicates that the device is offline.
  5. Find the interface name that was assigned to the device: 
    # cat /sys/bus/ccwgroup/drivers/qeth/0.0.f500/if_name
enccw0.0.f500
    Now you can continue to configure the enccw0.0.f500 network interface.
    The following command from the s390utils package shows the most important settings of your qeth device: 
    # lsqeth enccw0.0.f500
Device name                     : enccw0.0.f500
-------------------------------------------------
card_type               : OSD_1000
cdev0                   : 0.0.f500
cdev1                   : 0.0.f501
cdev2                   : 0.0.f502
chpid                   : 76
online                  : 1
portname                : OSAPORT
portno                  : 0
state                   : UP (LAN ONLINE)
priority_queueing       : always queue 0
buffer_count            : 16
layer2                  : 1
isolation               : none

20.3.1.2. Dynamically Removing a qeth Device

To remove a qeth device, use the znetconf utility. For example:
  1. Use the znetconf utility to show you all configured network devices: 
    # znetconf -c
Device IDs                 Type    Card Type      CHPID Drv. Name        	State
--------------------------------------------------------------------------------------
0.0.8036,0.0.8037,0.0.8038 1731/05 HiperSockets      FB qeth hsi1        	online
0.0.f5f0,0.0.f5f1,0.0.f5f2 1731/01 OSD_1000          76 qeth enccw0.0.09a0      online
0.0.f500,0.0.f501,0.0.f502 1731/01 GuestLAN QDIO     00 qeth enccw0.0.f500      online
  2. Select the network device to be removed and run znetconf to set the device offline and ungroup the ccw> group device. 
    # znetconf -r f500
Remove network device 0.0.f500 (0.0.f500,0.0.f501,0.0.f502)?
Warning: this may affect network connectivity!
Do you want to continue (y/n)?y
Successfully removed device 0.0.f500 (enccw0.0.f500)
  3. Verify the success of the removal: 
    # znetconf -c
Device IDs                 Type    Card Type      CHPID Drv. Name        	State
--------------------------------------------------------------------------------------
0.0.8036,0.0.8037,0.0.8038 1731/05 HiperSockets      FB qeth hsi1        	online
0.0.f5f0,0.0.f5f1,0.0.f5f2 1731/01 OSD_1000          76 qeth enccw0.0.09a0      online

20.3.1.3. Persistently Adding a qeth Device

To make your new qeth device persistent, you need to create the configuration file for your new interface. The network interface configuration files are placed in the /etc/sysconfig/network-scripts/ directory.
The network configuration files use the naming convention ifcfg-device, where device is the value found in the if_name file in the qeth group device that was created earlier, for example enccw0.0.09a0. The cio_ignore commands are handled transparently for persistent device configurations and you do not need to free devices from the ignore list manually.
If a configuration file for another device of the same type already exists, the simplest way is to copy it to the new name and then edit it: 
# cd /etc/sysconfig/network-scripts
# cp ifcfg-enccw0.0.09a0 ifcfg-enccw0.0.0600
To learn IDs of your network devices, use the lsqeth utility: 
# lsqeth -p
devices                    CHPID interface        cardtype       port chksum prio-q'ing rtr4 rtr6 lay'2 cnt
-------------------------- ----- ---------------- -------------- ---- ------ ---------- ---- ---- ----- -----
0.0.09a0/0.0.09a1/0.0.09a2 x00   enccw0.0.09a0    Virt.NIC QDIO  0    sw     always_q_2 n/a  n/a  1     64
0.0.0600/0.0.0601/0.0.0602 x00   enccw0.0.0600    Virt.NIC QDIO  0    sw     always_q_2 n/a  n/a  1     64
If you do not have a similar device defined, you must create a new file. Use this example of /etc/sysconfig/network-scripts/ifcfg-0.0.09a0 as a template: 
# IBM QETH
DEVICE=enccw0.0.09a0
BOOTPROTO=static
IPADDR=10.12.20.136
NETMASK=255.255.255.0
ONBOOT=yes
NETTYPE=qeth
SUBCHANNELS=0.0.09a0,0.0.09a1,0.0.09a2
PORTNAME=OSAPORT
OPTIONS='layer2=1 portno=0'
MACADDR=02:00:00:23:65:1a
TYPE=Ethernet
Edit the new ifcfg-0.0.0600 file as follows:
  1. Modify the DEVICE statement to reflect the contents of the if_name file from your ccw group.
  2. Modify the IPADDR statement to reflect the IP address of your new interface.
  3. Modify the NETMASK statement as needed.
  4. If the new interface is to be activated at boot time, then make sure ONBOOT is set to yes.
  5. Make sure the SUBCHANNELS statement matches the hardware addresses for your qeth device. Please, note that the IDs must be specified in lowercase.
  6. Modify the PORTNAME statement or leave it out if it is not necessary in your environment.
  7. You can add any valid sysfs attribute and its value to the OPTIONS parameter. The Red Hat Enterprise Linux installation program currently uses this to configure the layer mode (layer2) and the relative port number (portno) of qeth devices.
    The qeth device driver default for OSA devices is now layer 2 mode. To continue using old ifcfg definitions that rely on the previous default of layer 3 mode, add layer2=0 to the OPTIONS parameter.
/etc/sysconfig/network-scripts/ifcfg-0.0.0600 
# IBM QETH
DEVICE=enccw0.0.0600
BOOTPROTO=static
IPADDR=192.168.70.87
NETMASK=255.255.255.0
ONBOOT=yes
NETTYPE=qeth
SUBCHANNELS=0.0.0600,0.0.0601,0.0.0602
PORTNAME=OSAPORT
OPTIONS='layer2=1 portno=0'
MACADDR=02:00:00:b3:84:ef
TYPE=Ethernet
Changes to an ifcfg file only become effective after rebooting the system or after the dynamic addition of new network device channels by changing the system's I/O configuration (for example, attaching under z/VM). Alternatively, you can trigger the activation of a ifcfg file for network channels which were previously not active yet, by executing the following commands:
  1. Use the cio_ignore utility to remove the network channels from the list of ignored devices and make them visible to Linux: 
    # cio_ignore -r read_device_bus_id,write_device_bus_id,data_device_bus_id
    Replace read_device_bus_id,write_device_bus_id,data_device_bus_id with the three device bus IDs representing a network device. For example, if the read_device_bus_id is 0.0.0600, the write_device_bus_id is 0.0.0601, and the data_device_bus_id is 0.0.0602: 
    #  cio_ignore -r 0.0.0600,0.0.0601,0.0.0602
  2. To trigger the uevent that activates the change, issue: 
    # echo add > /sys/bus/ccw/devices/read-channel/uevent
    For example: 
    # echo add > /sys/bus/ccw/devices/0.0.0600/uevent
  3. Check the status of the network device: 
    # lsqeth
  4. Now start the new interface: 
    # ifup enccw0.0.0600
  5. Check the status of the interface: 
    # ip addr show enccw0.0.0600
3: enccw0.0.0600:  <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 3c:97:0e:51:38:17 brd ff:ff:ff:ff:ff:ff
inet 10.85.1.245/24 brd 10.34.3.255 scope global dynamic enccw0.0.0600
valid_lft 81487sec preferred_lft 81487sec
inet6 1574:12:5:1185:3e97:eff:fe51:3817/64 scope global noprefixroute dynamic
valid_lft 2591994sec preferred_lft 604794sec
inet6 fe45::a455:eff:d078:3847/64 scope link
valid_lft forever preferred_lft forever
  6. Check the routing for the new interface: 
    # ip route
default via 10.85.1.245 dev enccw0.0.0600  proto static  metric 1024
12.34.4.95/24 dev enp0s25  proto kernel  scope link  src 12.34.4.201
12.38.4.128 via 12.38.19.254 dev enp0s25  proto dhcp  metric 1
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1
  7. Verify your changes by using the ping utility to ping the gateway or another host on the subnet of the new device: 
    # ping -c 1 192.168.70.8
PING 192.168.70.8 (192.168.70.8) 56(84) bytes of data.
64 bytes from 192.168.70.8: icmp_seq=0 ttl=63 time=8.07 ms
  8. If the default route information has changed, you must also update /etc/sysconfig/network accordingly.

20.3.2. Adding an LCS Device

The LAN channel station (LCS) device driver supports 1000Base-T Ethernet on the OSA-Express2 and OSA-Express 3 features.
The LCS device driver assigns the following interface name for OSA-Express Fast Ethernet and Gigabit Ethernet devices: enccwbus_ID. The bus ID is composed of the channel subsystem ID, subchannel set ID, and device number, for example enccw0.0.0a00.

20.3.2.1. Dynamically Adding an LCS Device

  1. Load the device driver: 
    # modprobe lcs
  2. Use the cio_ignore utility to remove the network channels from the list of ignored devices and make them visible to Linux: 
    # cio_ignore -r read_device_bus_id,write_device_bus_id
    Replace read_device_bus_id and write_device_bus_id with the two device bus IDs representing a network device. For example: 
    # cio_ignore -r 0.0.09a0,0.0.09a1
  3. Create the group device: 
    # echo read_device_bus_id,write_device_bus_id > /sys/bus/ccwgroup/drivers/lcs/group
  4. Configure the device. OSA cards can provide up to 16 ports for a single CHPID. By default, the LCS group device uses port 0. To use a different port, issue a command similar to the following: 
    # echo portno > /sys/bus/ccwgroup/drivers/lcs/device_bus_id/portno
    Replace portno with the port number you want to use.
  5. Set the device online: 
    # echo 1 > /sys/bus/ccwgroup/drivers/lcs/read_device_bus_id/online
  6. To find out what network device name has been assigned, enter the command: 
    # ls -l /sys/bus/ccwgroup/drivers/lcs/read_device_bus_ID/net/
drwxr-xr-x 4 root root 0 2010-04-22 16:54 enccw0.0.0600

20.3.2.2. Persistently Adding an LCS Device

The cio_ignore commands are handled transparently for persistent device configurations and you do not need to free devices from the ignore list manually.
To add an LCS device persistently, follow these steps:
  1. Create a configuration script as file in /etc/sysconfig/network-scripts/ with a name like ifcfg-device, where device is the value found in the if_name file in the qeth group device that was created earlier, for example enccw0.0.09a0. The file should look similar to the following: 
    # IBM LCS
DEVICE=enccw0.0.09a0
BOOTPROTO=static
IPADDR=10.12.20.136
NETMASK=255.255.255.0
ONBOOT=yes
NETTYPE=lcs
SUBCHANNELS=0.0.09a0,0.0.09a1
PORTNAME=0
OPTIONS=''
TYPE=Ethernet
  2. Modify the value of PORTNAME to reflect the LCS port number (portno) you would like to use. You can add any valid lcs sysfs attribute and its value to the optional OPTIONS parameter. See Section 20.3.1.3, “Persistently Adding a qeth Device” for the syntax.
  3. Set the DEVICE parameter as follows: 
    DEVICE=enccwbus_ID
  4. Issue an ifup command to activate the device: 
    # ifup enccwbus_ID
Changes to an ifcfg file only become effective after rebooting the system. You can trigger the activation of a ifcfg file for network channels by executing the following commands:
  1. Use the cio_ignore utility to remove the LCS device adapter from the list of ignored devices and make it visible to Linux: 
    # cio_ignore -r read_device_bus_id,write_device_bus_id
    Replace read_device_bus_id and write_device_bus_id with the device bus IDs of the LCS device. For example: 
    # cio_ignore -r 0.0.09a0,0.0.09a1
  2. To trigger the uevent that activates the change, issue: 
    # echo add > /sys/bus/ccw/devices/read-channel/uevent
    For example: 
    # echo add > /sys/bus/ccw/devices/0.0.09a0/uevent

20.3.3. Configuring a IBM Z Network Device for Network Root File System

To add a network device that is required to access the root file system, you only have to change the boot options. The boot options can be in a parameter file (see Chapter 21, Parameter and Configuration Files on IBM Z) or part of a zipl.conf on a DASD or FCP-attached SCSI LUN prepared with the zipl boot loader. There is no need to recreate the initramfs.
Dracut, the mkinitrd successor that provides the functionality in the initramfs that in turn replaces initrd, provides a boot parameter to activate network devices on IBM Z early in the boot process: rd.znet=.
As input, this parameter takes a comma-separated list of the NETTYPE (qeth, lcs, ctc), two (lcs, ctc) or three (qeth) device bus IDs, and optional additional parameters consisting of key-value pairs corresponding to network device sysfs attributes. This parameter configures and activates the IBM Z network hardware. The configuration of IP addresses and other network specifics works the same as for other platforms. See the dracut documentation for more details.
The cio_ignore commands for the network channels are handled transparently on boot.
Example boot options for a root file system accessed over the network through NFS: 
root=10.16.105.196:/nfs/nfs_root cio_ignore=all,!condev rd.znet=qeth,0.0.0a00,0.0.0a01,0.0.0a02,layer2=1,portno=0,portname=OSAPORT ip=10.16.105.197:10.16.105.196:10.16.111.254:255.255.248.0:nfs‑server.subdomain.domain:enccw0.0.09a0:none rd_NO_LUKS rd_NO_LVM rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us