Why does value for busy threads in 'th'(thread) line remains zero in /proc/net/rpc/nfsd?

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • NFS server

Issue

  • Why does value for busy_threads remains zero in /proc/net/rpc/nfsd?
  • NFS server appears busy but busy_threads remains zero?
  • Not able to identify for how much time threads were busy.
th 8 0 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
  • nfsd thread count / nfs thread count

Resolution

  • This feature was removed in an upstream patch in kernel version 2.6.29and RHEL-6 is based on kernel 2.6.32.

  • Refer knfsd: remove the nfsd thread busy histogram for more details on the patch.

  • There are no ways to get a similar histogram in Red Hat Enterprise 6.

Root Cause

  • The NFSD thread busy histogram feature was discarded because of these three major reasons:

    • (1) Function update_thread_usage() which is used to calculate the data for threads line, increments buckets by call service time or call arrival time in jiffies. On lightly loaded machines, call service times are usually < 1 jiffy. And on heavily loaded machines call arrival times will be << 1 jiffy. So a large portion of the updates to the buckets are rounded down to zero, and the histogram is undercounting.
    • (2) The format in which the histogram is presented is cryptic, which is difficult to explain, and difficult to use.
    • (3) Updating the histogram requires taking a global spinlock and dirtying the global variables nfsd_last_call, nfsd_busy, and nfsdstats twice on every RPC call, which is a significant scaling limitation.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments