RHEL8: NFSv4 + fscache always writes to cache but never reads back from the cache

Issue

• An NFSv4 mount with fscache enabled never reads back from the cache
• We can see this is we mount the share, use 'dd' to read a file from the share, drop caches to drop the page cache, then use 'dd' again to read the file a second time. On the second 'dd' there should be no NFS operations recorded, since the reads should come from fscache not over the wire. Instead there is 5 reads again, the same amount as after the first 'dd'.

rhel8 # mount -o vers=4,fsc nfsserver:/export /mnt/nfsserver
rhel8 # dd if=/mnt/nfsserver/file of=/dev/null
rhel8 # grep READ: /proc/self/mountstats
            READ: 5 5 0 1120 1049096 0 2 4
rhel8 # echo 3 >/proc/sys/vm/drop_caches
rhel8 # dd if=/mnt/nfsserver/file of=/dev/null
rhel8 # grep READ: /proc/self/mountstats
            READ: 10 10 0 2240 2098192 5 8 15