Processes accessing GFS2 hanging or freezing

Issue

• Processes accessing a GFS2 file system or file systems are "hanging" or "freezing".
• Processes that are in this hung state show as being in "D" state in ps output.
• On RHEL 5.5 or higher backtrace begins to appear in the messages logs indicating that processes have hung for more than 120 seconds.
• The processes being logged are the processes attempting to access GFS2.
• No matter how much time is allowed to pass the file system never returns to an operational state.
• The only workaround is to reboot one or more of the nodes in the cluster
• The hangs hapen with some regularity