Changes to Infiniband drivers in RHEL 7.4

Posted on

It appears that there were some significant changes to the built-in Infiniband drivers in RHEL 7.4. Among the changes, the "libmlx4" package has been deprecated and replaced by "libibverbs". I had some custom tuning settings that no longer work.
Specifically I was setting "log_num_mtt" in a file in /etc/modprobe.d/ and that option is no longer supported apparently.
It looks like "log_mtts_per_seg" is supported, but a value of "21" is no accepted (that's what I was using for "log_num_mtt", assuming "log_mtts_per_seg" was zero).

Are these changes documented anywhere?

According to the discussion linked below on the OpenMPI FAQ page, the parameters should be used like this:
max_reg_mem = (2^log_num_mtt) * (2^log_mtts_per_seg) * PAGE_SIZE

https://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages-more

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.