How can I diagnose the cause of scsi reservation conflicts in a RHEL cluster using fence_scsi?
Issue
- Cluster node logs "scsi reservation conflict" during bootup and shared storage is inaccessible or GFS/GFS2 file system withdraws:
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 4:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: sd 2:0:0:9: reservation conflict
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: fatal: I/O error
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: block = 86382275
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: function = gfs_logbh_wait
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: file = /builddir/build/BUILD/gfs-kmod-0.1.34/_kmod_build_/src/gfs/dio.c, line = 816
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: time = 1359814274
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: about to withdraw from the cluster
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: telling LM to withdraw
kernel: GFS: fsid=isgruapp7n-prod:amgprod_gfs.0: withdrawn
- The cluster nodes are reporting reservation conflicts errors:
kernel: sd 2:0:5:0: reservation conflict
kernel: sd 2:0:5:0: SCSI error: return code = 0x00000018
kernel: end_request: I/O error, dev sdq, sector 423224328
kernel: device-mapper: multipath: Failing path 65:0.
kernel: sd 2:0:5:0: reservation conflict
kernel: sd 2:0:5:0: SCSI error: return code = 0x00000018
kernel: end_request: I/O error, dev sdq, sector 423224336
- GFS mount issue on 2 node RH cluster. Mounting GFS filesystems:
/sbin/mount.gfs: error mounting /dev/mapper/SMC_VG-smcdata on /isg/smc: No such file or directory
Environment
- Red Hat Enterprise Linux (RHEL); including
- Red Hat Enterprise Linux 5 (with the High Availability or Resilient Storage Add On)
- Red Hat Enterprise Linux 6 (with the High Availability or Resilient Storage Add On)
- Red Hat High Availability Cluster with 2 or more nodes.
- One or more cluster nodes configured to use
fence_scsi
in/etc/cluster/cluster.conf
:
$ grep fence_scsi /etc/cluster/cluster.conf
<fencedevice agent="fence_scsi" name="myfencing"/>
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.