df gives wrong output of bind mounts on the NFS share

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 6
  • nfs

Issue

  • df shows incorrect mount point and device name for the bind mounts however, the utilization is correct.
  • The issue is observed only for shares exported from the same NFS server.
# cat /proc/mounts | grep -i nfs
server:/testnfs/ /testnfs nfs4 rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=Y.Y.Y.Y,minorversion=0,local_lock=none,addr=X.X.X.X 0 0
server:/testnfs2/ /testnfs1 nfs4 rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=Y.Y.Y.Y,minorversion=0,local_lock=none,addr=X.X.X.X 0 0
server:/testnfs/ /bind nfs4 rw,relatime,vers=4,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=Y.Y.Y.Y,minorversion=0,local_lock=none,addr=X.X.X.X 0 0

# df -h /bind
Filesystem        Size  Used Avail Use% Mounted on
/testnfs/testdir   20G  4.7G   15G  24% /bind

# df -h /bind/
Filesystem            Size  Used Avail Use% Mounted on
server:/testnfs2
                       20G  4.7G   15G  24% /testnfs1                           << incorrect device and mount point

Resolution

The workaround is to mount the shares coming from the same nfs server either with different options or different nfs versions to make them unique.

Root Cause

  • The device number of the nfs shares should be unique, the same can be verified from /proc/self/mountinfo.
  • In our case, the device numbers are same hence when df is executed against the bind mount, it refers the last entry present in /etc/mtab.

Diagnostic Steps

  • The device numbers for the NFS shares are the same, 0:20 per /proc/self/mountinfo
# cat /proc/self/mountinfo | grep -i nfs4 | awk '{print $3,$5,$9 }' | sort -n
0:20 /bind server:/testnfs/
0:20 /testnfs1 server:/testnfs2/
0:20 /testnfs server:/testnfs/
  • df for /bind/ refers the last entry from /etc/mtab and displays it in the output
# cat /etc/mtab | grep -i nfs
server:/testnfs /testnfs nfs rw,vers=4,addr=X.X.X.X,clientaddr=Y.Y.Y.Y 0 0
server:/testnfs2 /testnfs1 nfs rw,vers=4,addr=X.X.X.X,clientaddr=Y.Y.Y.Y 0 0          << << <<
/testnfs/testdir /bind none rw,bind 0 0
  • When the last entry of the mounted NFS per the /etc/mtab is unmounted then it shows the correct device name in df
# df -hP /bind
Filesystem        Size  Used Avail Use% Mounted on
/testnfs/testdir   20G  4.7G   15G  24% /bind

# df -hP /bind/
Filesystem             Size  Used Avail Use% Mounted on
server:/testnfs2   20G  4.7G   15G  24% /testnfs1

# umount /testnfs1

# df -hP /bind/
Filesystem            Size  Used Avail Use% Mounted on
server:/testnfs   20G  4.7G   15G  24% /testnfs

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.