Show Table of Contents
Red Hat Training
A Red Hat training course is available for Red Hat Enterprise Linux
13.4. InfiniBand and RDMA related software packages
Because RDMA applications are so different from Berkeley Sockets based applications, and from normal
IP networking, most applications that are used on an IP network cannot be used directly on an RDMA network. Red Hat Enterprise Linux 7 comes with a number of different software packages for RDMA network administration, testing and debugging, high level software development APIs, and performance analysis.
In order to utilize these networks, some or all of these packages need to be installed (this list is not exhaustive, but does cover the most important packages related to RDMA).
Required packages:
rdma— responsible for kernel initialization of the RDMA stack.libibverbs— provides the InfiniBand Verbs API.opensm— subnet manager (only required on one machine, and only if there is no subnet manager active on the fabric).user space driver for installed hardware— one or more of: infinipath-psm, libcxgb3, libcxgb4, libehca, libipathverbs, libmthca, libmlx4, libmlx5, libnes, and libocrdma. Note that libehca is only available for IBM Power Systems servers.
Recommended packages:
librdmacm, librdmacm-utils, and ibacm— Connection management library that is aware of the differences between InfiniBand, iWARP, and RoCE and is able to properly open connections across all of these hardware types, some simple test programs for verifying the operation of the network, and a caching daemon that integrates with the library to make remote host resolution faster in large clusters.libibverbs-utils— Simple Verbs based programs for querying the installed hardware and verifying communications over the fabric.infiniband-diags and ibutils— Provide a number of useful debugging tools for InfiniBand fabric management. These provide only very limited functionality on iWARP or RoCE as most of the tools work at the InfiniBand link layer, not the Verbs API layer.perftest and qperf— Performance testing applications for various types of RDMA communications.
Optional packages:
These packages are available in the Optional channel. Before installing packages from the Optional channel, see Scope of Coverage Details. Information on subscribing to the Optional channel can be found in the Red Hat Knowledgebase solution How to access Optional and Supplementary channels.
dapl, dapl-devel, and dapl-utils— Provide a different API for RDMA than the Verbs API. There is both a runtime component and a development component to these packages.openmpi, mvapich2, and mvapich2-psm— MPI stacks that have the ability to use RDMA communications. User-space applications writing to these stacks are not necessarily aware that RDMA communications are taking place.