RHEL6.9 / RHEL7: NFS client with TCP timestamps disabled sees periodic 60 seconds of unavailability due to EADDRNOTAVAIL response from kernel_connect during TCP TIME_WAIT state

Solution Verified - Updated -

Issue

  • NFS client on VMware guest, through NAT, or in low-latency environments have TCP timestamps disabled (net.ipv4.tcp_timestamps=0) and seeing 60 seconds of periodic, temporary hangs / unavailability of NFS shares
  • After updating to RHEL6.9 kernel, seeing periods of high CPU usage of rpciod kernel threads on NFS client machines
  • excessively high 'retrans' number from 'nfsstat -r' output after updating to RHEL6.9 kernel
  • Jobs are failing while trying to access the NFS share and we could see the errors mentioned below in the log file. We use 'soft' mount option and low 'timeo' value and are facing this issue since patching the server to RHEL6.9 kernel:
Apr 29 23:01:08 nfs-client kernel: nfs: server server.domain.com not responding, timed out
Apr 29 23:01:12 nfs-client kernel: nfs: server server.domain.com not responding, timed out
Apr 29 23:01:17 nfs-client kernel: nfs: server server.domain.com not responding, timed out
Apr 29 23:01:21 nfs-client kernel: nfs: server server.domain.com not responding, timed out
Apr 29 23:01:25 nfs-client kernel: nfs: server server.domain.com not responding, timed out

Environment

  • Red Hat Enterprise Linux (RHEL) 6.9 (NFS client)
    • kernel between 2.6.32-696.el6 and before kernel-2.6.32-696.10.1.el6
  • Red Hat Enterprise Linux 7 (NFS client)
    • kernel between 3.10.0-514.el7 and before 3.10.0-514.35.1.el7
    • kernel between 3.10.0-693.el7 and before 3.10.0-693.5.2.el7
  • either NFSv3 or NFSv4
  • NFS over TCP, with TCP timestamps option disabled on the connection (see RFC 1323)
    • On Linux, net.ipv4.tcp_timestamps = 0 on either NFS client or server is sufficient
  • NFS TCP connection disconnect / reconnect sequence due to idle timer or some other NFS error (but not explicit umount).

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.