Red Hat Training

A Red Hat training course is available for Red Hat Enterprise Linux

4.115. OFED

Updated OpenFabrics Alliance (openib) packages that upgrade the OpenFabrics Enterprise Distribution (OFED) stack, fix multiple bugs and add various enhancements are now available for Red Hat Enterprise Linux 5.
The OpenFabrics Enterprise Distribution (OFED) is a collection of InfiniBand and iWARP hardware diagnostic utilities, the InfiniBand fabric management daemon, the Infiniband and iWARP kernel module loader, as well as libraries and development packages for writing applications that use Remote Direct Memory Access (RDMA) technology. Red Hat Enterprise Linux uses the OFED software stack as its complete stack for InfiniBand, iWARP, and RDMA hardware support.


The InfiniBand driver stack in the Red Hat Enterprise Linux 5.9 kernel has been updated to the OpenFabrics Enterprise Distribution (OFED) stack, and this erratum updates the user space packages to work with the updated kernel. The user space packages are updated to the same level as the Red Hat Enterprise Linux 6.3 user space InfiniBand software stack with the exception that in Red Hat Enterprise Linux 5.9 we do not support InfiniBand over Ethernet (IBoE), also known as RDMA over Converged Ethernet (RoCE).

Bug Fixes

IP over InfinBand (IPoIB) interfaces are artificial constructs created on top of InfiniBand RDMA devices. The IPoIB interface has a generated hardware MAC address. The queue pair that the interface is attached to is encoded in the hardware MAC address. However, when unloading and reloading the IPoIB module, it is likely that a different queue pair to the IPoIB queue pair will be assigned. Because the queue pair number is encoded in the MAC address of the IPoIB interface, and because it can change when the IPoIB module is unloaded and reloaded, the final MAC address of IPoIB interfaces can change. Consequently, when users created ifcfg-ibX files that specified the MAC address of the IPoIB interface, after a reload of the IPoIB kernel module, when the MAC address no longer matched, the IPoIB interface failed to be recognized as a configured interface by the network subsystem. To solve this problem, this update implements custom ifup-ib and ifdown-ib network scripts that are aware of the portion of an IPoIB's MAC address that is constant versus the portion that is subject to change. As a result, when a user reloads the IPoIB module, and when the IPoIB interface's MAC address changes, the ifup-ib and ifdown-ib scripts will properly match against the portion that did not change and recognize the interface correctly.
Imperfect argument processing in the ibv_rc_pingpong program allowed arguments that were too large to be passed to the program. Consequently, the program terminated unexpectedly with a segmentation fault when attempting to set up transfers using large arguments. The checking of arguments in ibv_rc_pingpong has been strengthened. As a result, the program no longer crashes on bad arguments.
Insufficient state checking in the ifup-ib script could cause it to create an invalid state on bond devices that had IPoIB slaves. Consequently, when the user called ifup on the IPoIB interface, and expected it to be working, the master bond device was sometimes taken down. A check to make sure that the device is not already present in the bond device before attempting to add it is now made. As a result, the device is now initialized properly.
The libibverbs.spec file was missing the BuildRequires: valgrind-devel line. Consequently, the libibverbs package was built without valgrind memory allocation debugging support. The required line has been added to the spec file and the libibverbs package now supports valgrind memory debugging.
When passed the -r flag, the ibdiagnet program attempted to free the same memory twice. Consequently, the program would trigger protection built into glibc and end the execution. The program has been fixed to no longer attempt to free the same memory twice and as a result the program completes as expected.
The ibnodes program is a simple shell wrapper script that calls ibhosts and ibswitches. When passed the -h switch to get help for the program, it passed that switch on to both ibhosts and ibswitches. Consequently, when the user ran ibnodes -h they saw help output for ibhosts and ibswitches. A simple help handler in the ibnodes program that outputs ibnodes-specific help information has been implemented and the problem no longer occurs.
A race condition on handling of completion events could confuse the ib_send_lat test program. Consequently, the ib_send_lat test program sometimes terminated unexpectedly with a segmentation fault. Completion processing has now been separated into two separate queues, one for send completions and one for receive completions. As a result, out of order and unexpected completions no longer confuse the test program, nor cause crashes.
An error in thread handling resulted in the rping binary freeing resources before all threads that accessed those resources had exited. Consequently, the rping binary terminated unexpectedly with a segmentation fault when attempting to access already freed resources. Thread handling has been improved to wait for all threads to exit in various locations before proceeding with freeing memory resources. As a result, the rping application no longer crashes on iWARP hardware in the scenario described.
The openibd init script loaded the parent RDS module if configured to do so, but did not load either of the RDS transports (TCP and RDMA). RDS is non-functional without at least one transport module loaded. Consequently, the RDS protocol was listed as available, but was not usable because no suitable interfaces with a supported transport could be found. The openibd init script has been updated to always load the TCP and RDMA transports if the RDS service is configured to be enabled in /etc/ofed/ofed.conf. As a result, the RDS protocol is now functional and can find suitable interfaces over which to operate.
Early versions of the Qperf application had the value for the RDS address family protocol value hardcoded because it was not specified via the normal system headers at the time. When the RDS protocol was accepted upstream, the preliminary address family value was changed. Qperf did not pick this change up and instead used the incorrect constant. Consequently, when Qperf thought it was attempting to open an RDS protocol socket, it was in fact attempting to open a different, unsupported socket family. This updates removes the old preliminary constant from the qperf sources and instead gets the constant from the kernel headers now that it is included. As a result, Qperf will open the correct socket type and operate as expected.
The ifup-ib script did not support naming an IPoIB interface anything other than ib0 or ib1. Consequently, it was not possible to give more meaningful names to IPoIB interfaces. The ifup-ib script has been updated to match against the MAC address in an ifcfg-[name] file, and if the name of the device as specified in the file is not the same as the name of the device according to the kernel, then it will use the iproute tools to rename the device in the kernel. As a result, it is now possible to create a file ifcfg-mlx4_1 with a device HWADDR=[MAC address of mlx4_1 port], and DEVICE=[desired device name] and have the ifup-ib scripts automatically rename the device to the desired device name.
Reuse of the outgoing RDS ping sockets before the last sent package had received its ACK caused the rds-ping utility to corrupt its internal data. This happened anytime the user specified a ping interval short enough to result in there being 8 or more outstanding ping packets as there are 8 outgoing sockets used by rds-ping by default. Given that RDS ping times are often in the 20-30 msec range, any value less than 8 or so msec for the ping interval ran a reasonable chance of causing this problem. Consequently, the rds-ping utility would give erratic results and sometimes terminate unexpectedly with a segmentation fault. The rds-ping send path has been reworked to track whether or not the response packet has been received on each socket. When it is time to send the next ping, if all sockets still have outstanding pings lacking a response, the program now waits until a ping response comes back in and frees up a socket for sending again. As a result, the application now works in a reliable manner when given very short ping intervals.
The rping program tried to give an invalid value for the retry_count element on newly created queue pairs. Consequently, depending on the particular hardware driver in question, the driver would either refuse to create the queue pair, causing rping to fail to create a connection and exit, or it would silently truncate the out-of-bounds value to a smaller value and proceed with the connection. This update changes rping to request a valid value for retry_count. As a result, regardless of hardware the connection is now accepted and works as expected.


The mstflint package, which provides Mellanox firmware burning and diagnostics tools, now includes support for Mellanox ConnectX-3 devices.
The libcxgb3 library has been updated from upstream version 1.3.0 to the latest upstream version, version 1.3.1. The libcxgb3 library provides a user space driver for Chelsio cxgb3 hardware to be used with libibverbs. This update refreshes the driver for enhanced hardware support and bug fixes.
The libcxgb4 library has been updated from upstream version 1.1.1 to the latest upstream version, version 1.2.0. The libcxgb4 library provides a user space driver for Chelsio cxgb4 hardware to be used with libibverbs. This update refreshes the driver for enhanced hardware support and bug fixes.
All users using Infiniband and iWARP hardware are advised to upgrade to these updated openib packages, which fix these bugs and add these enhancements.