RHEL8: fscache is not respected after an unmount and mount of the NFS filesystem as compared to RHEL7

Issue

• On RHEL7, we can do the following:

rhel7 # systemctl start cachefilesd
rhel7 # mount -o fsc nfsserver:/export /mnt/nfsserver
rhel7 # dd if=/mnt/nfsserver/file of=/dev/null
rhel7 # reboot
rhel7 # systemctl start cachefilesd
rhel7 # mount -o vers=3,fsc nfsserver:/export /mnt/nfsserver
rhel7 # dd if=/mnt/nfsserver/file of=/dev/null

• After a reboot, the cached file is successfully read from local storage (/var/cache/fscache).
• On rhel8, the file is never read from the disk cache, instead it writes to disk again. It is a similar result if you unmount and remount the filesystem - no reads from cache.