RHEL6.7.z: NFS client with kernels 2.6.32-573.10.2.el6 or above hangs with 'not responding, still trying' messages and running processes in _spin_lock

Solution In Progress - Updated -

Issue

Seeing BUG: soft lockup messages with a process stuck in _spin_lock called from nfs_clear_request_commit
NFS client with flush thread stuck in _spin_lock called from nfs_scan_commit
NFS client with many running processes stuck in _spin_lock, and nfs: server XYZ not responding, still trying seen

Several processes are kept in D state (disk sleep) and never returns. A "df" command never retuns, and hangs. Even a "sudo" command does not work. Load average keep increasing with no process consuming high CPU. Ultimately we had to reboot the server.

When the problem strikes, we see some of these messages:

Feb 25 10:27:28 nfs-client kernel: [64627.406889] lockd: server nfs-server.example.com not responding, still trying

Environment

  • Red Hat Enterprise Linux 6.7 (NFS client)
    • kernels 2.6.32-573.10.2.el6 or above and earlier than 2.6.32-573.27.1.el6
    • kernel earlier than 2.6.32-642.el6
  • NFS

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In