rport-1:0-0: blocked FC remote port time out: saving binding

Issue

• 系统 /var/log/messages、/var/log/dmesg 或 vmcore 包含以下记录的事件，表示主机 HBA 和存储之间的 FC 路径丢失：

  {RHEL6,7}
      Jun  5 23:46:53 HOST kernel:  rport-1:0-0: blocked FC remote port time out: removing target and saving binding
      Jun  5 23:46:53 HOST kernel:  rport-1:0-1: blocked FC remote port time out: removing target and saving binding
  
  {RHEL5}
      Jun  5 23:46:53 HOST kernel:  rport-1:0-0: blocked FC remote port time out: saving binding
      Jun  5 23:46:53 HOST kernel:  rport-1:0-1: blocked FC remote port time out: saving binding
  
     -or if option fc_remove_on_dev_loss set-
      Jun  5 23:46:53 HOST kernel:  rport-1:0-0: blocked FC remote port time out: removing target and saving binding
      Jun  5 23:46:53 HOST kernel:  rport-1:0-1: blocked FC remote port time out: removing target and saving binding
  

• 设备映射器多路径路径出现问题，multipath -ll 显示路径停掉了：

  360060e80056204000000620400000085 dm-1 HP,OPEN-V
  [size=30G][features=1 queue_if_no_path][hwhandler=0][rw]
   \_ round-robin 0 [prio=2][active]
   \_ 2:0:0:1 sdb 8:16  [active][ready]
   \_ 3:0:0:1 sdd 8:48  [failed][ready]   <---
                         ^^^^^^
  

• 在 /var/log/messages 中发现记录了以下消息，造成这个问题的可能的原因和解决方案是什么：

  Jan 20 15:54:20 HOST kernel: qla2xxx 0000:0f:00.1: LOOP DOWN detected (2 3 0).
  Jan 20 15:54:36 HOST kernel:  rport-3:0-0: blocked FC remote port time out: saving binding
  Jan 20 15:54:36 HOST kernel: sd 3:0:0:1: SCSI error: return code = 0x00010000
  Jan 20 15:54:36 HOST kernel: end_request: I/O error, dev sdd, sector 593496
  Jan 20 15:54:36 HOST kernel: device-mapper: multipath: Failing path 8:48.
  

• Ext4 文件系统挂载变为只读，并导致系统挂起，在消息中观察到 blocked FC remote port time out: saving binding。

• 服务器内核 panic ，且 vmcore 包含阻止的 FC 远程端口错误消息，什么导致了这个问题？