NodeFilesystemSpaceFillingUp alert troubleshooting in OpenShift Container Platform 4

Issue

NodeFilesystemSpaceFillingUp alert is firing in the cluster

This alert is based on an extrapolation of the space used in a file system. It fires if both the current usage is above a certain threshold and the extrapolation predicts to run out of space in a certain time. This is a warning-level alert if that time is less than 24h. It's a critical alert if that time is less than 4h.

A filesystem running full is very bad for any process in need to write to the filesystem. But even before a filesystem runs full, performance is usually degrading.