Tools that poll for networking information are slow (ie: ifconfig, ip, net-snmp...)
Environment
Red Hat Enterprise Linux 6.6 (kernel 2.6.32-504.el6)
SFP+ capable Intel network cards
Issue
Running commands to get networking related information takes long time ( >1 second) to provide the output.
This is particularly evident in case of large number of interfaces, and it can be observed with the following:
# time ifconfig |grep -i "encap.*Ethernet" | wc -l
37
real 0m1.244s
user 0m0.001s
sys 0m0.008s
Resolution
Soluiton: The issue was fixed in Red Hat Enterprise Linux 6.6 kernel 2.6.32-504.23.4.el6, so update to this kernel.
Workaround 1: The bug is triggered by loading the driver when one or more network cards have no SFP+ module mounted, so, if you have related hardware (SFP+ modules) you can just insert at least one SFP+ module in each ixgbe NIC.
Workaround 2: Blacklist buggy [ixgbe] driver from loading.
Root Cause
The observed behaviour is due to a bug in the ixgbe driver.
The bug is triggered by loading the driver when 1 or more cards have no SFP+ module mounted. In this case the driver would not recognize the card as SFP+ capable triggering the call to identify_sfp in ixgbe_get_settings in order to verify the correctness of the identification.
If the SFP+ module is not present, the call would wait for the I2C reads timeout, causing the observed delay.
Diagnostic Steps
You can compare the [ixgbe] module code with the upstream patch and code of RHEL kernel released to be sure the patch is incuded:
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments